2024-09-05T00:58:29.172 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T00:58:29.198 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T00:58:29.270 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7889754' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1743 sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 sleep_before_teardown: 0 subset: 459/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsgPx/YCbwHpYN7om3E8KKAhjZWMCxtMPoczslGSeypx6mQSuG6LDCfeui2WeeSDMH/ukgenI/o8NweosUON+Y= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOgYQklT31Pf/ywdu3T2cLBJRkys6X7QqPvSn+F+BTAD+gUwZjdVNfcIWjAqPcztbyj6NjQr9JtTSDa7OALa2aY= smithi194.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXm0rX7JkXepIb4saktNVzCNbJCyjTQqGQ+JDlnx6DUdJ6cFTw6vZAem2ISxA0TRSiaB26mjMp3kGKmTXDQv6E= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-04_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T00:58:29.270 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa; will attempt to use it 2024-09-05T00:58:29.271 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/qa/tasks 2024-09-05T00:58:29.271 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T00:58:29.273 INFO:teuthology.task.internal:Checking packages... 2024-09-05T00:58:29.295 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4' 2024-09-05T00:58:29.295 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T00:58:29.295 INFO:teuthology.packaging:ref: None 2024-09-05T00:58:29.295 INFO:teuthology.packaging:tag: None 2024-09-05T00:58:29.295 INFO:teuthology.packaging:branch: squid 2024-09-05T00:58:29.295 INFO:teuthology.packaging:sha1: 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T00:58:29.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-05T00:58:29.494 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-259.g91c8a7e5 2024-09-05T00:58:29.496 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T00:58:29.502 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T00:58:29.502 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T00:58:29.514 INFO:teuthology.task.internal:Saving configuration 2024-09-05T00:58:29.535 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T00:58:29.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T00:58:29.559 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 00:53:38.506948', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNsgPx/YCbwHpYN7om3E8KKAhjZWMCxtMPoczslGSeypx6mQSuG6LDCfeui2WeeSDMH/ukgenI/o8NweosUON+Y='} 2024-09-05T00:58:29.576 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 00:53:38.503268', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOgYQklT31Pf/ywdu3T2cLBJRkys6X7QqPvSn+F+BTAD+gUwZjdVNfcIWjAqPcztbyj6NjQr9JtTSDa7OALa2aY='} 2024-09-05T00:58:29.593 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi194.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 00:53:38.505224', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCXm0rX7JkXepIb4saktNVzCNbJCyjTQqGQ+JDlnx6DUdJ6cFTw6vZAem2ISxA0TRSiaB26mjMp3kGKmTXDQv6E='} 2024-09-05T00:58:29.594 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T00:58:29.600 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-05T00:58:29.600 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-05T00:58:29.600 INFO:teuthology.task.internal:roles: ubuntu@smithi194.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-05T00:58:29.600 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T00:58:29.672 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff91410fbe0>, signals=[15]) 2024-09-05T00:58:29.672 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T00:58:29.677 INFO:teuthology.task.internal:Opening connections... 2024-09-05T00:58:29.678 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T00:58:29.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T00:58:29.756 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-09-05T00:58:29.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T00:58:29.832 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi194.front.sepia.ceph.com 2024-09-05T00:58:29.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T00:58:29.908 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T00:58:29.915 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-05T00:58:29.933 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-05T00:58:29.934 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-05T00:58:29.991 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T00:58:29.992 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T00:58:29.993 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-05T00:58:30.014 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-09-05T00:58:30.032 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-09-05T00:58:30.033 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-09-05T00:58:30.091 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T00:58:30.092 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T00:58:30.092 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-09-05T00:58:30.114 DEBUG:teuthology.orchestra.run.smithi194:> uname -m 2024-09-05T00:58:30.133 INFO:teuthology.orchestra.run.smithi194.stdout:x86_64 2024-09-05T00:58:30.133 DEBUG:teuthology.orchestra.run.smithi194:> cat /etc/os-release 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:NAME="CentOS Stream" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION="9" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:ID="centos" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:ID_LIKE="rhel fedora" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:VERSION_ID="9" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:PLATFORM_ID="platform:el9" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:ANSI_COLOR="0;31" 2024-09-05T00:58:30.190 INFO:teuthology.orchestra.run.smithi194.stdout:LOGO="fedora-logo-icon" 2024-09-05T00:58:30.191 INFO:teuthology.orchestra.run.smithi194.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T00:58:30.191 INFO:teuthology.orchestra.run.smithi194.stdout:HOME_URL="https://centos.org/" 2024-09-05T00:58:30.191 INFO:teuthology.orchestra.run.smithi194.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T00:58:30.191 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T00:58:30.191 INFO:teuthology.orchestra.run.smithi194.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T00:58:30.191 INFO:teuthology.lock.ops:Updating smithi194.front.sepia.ceph.com on lock server 2024-09-05T00:58:30.206 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T00:58:30.215 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T00:58:30.221 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T00:58:30.221 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-05T00:58:30.223 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-09-05T00:58:30.226 DEBUG:teuthology.orchestra.run.smithi194:> test '!' -e /home/ubuntu/cephtest 2024-09-05T00:58:30.246 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T00:58:30.252 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T00:58:30.252 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-05T00:58:30.281 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-09-05T00:58:30.283 DEBUG:teuthology.orchestra.run.smithi194:> test -z $(ls -A /var/lib/ceph) 2024-09-05T00:58:30.387 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T00:58:30.458 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T00:58:30.471 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-05T00:58:30.472 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-05T00:58:30.472 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T00:58:30.473 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T00:58:30.473 DEBUG:teuthology.orchestra.run.smithi194:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T00:58:30.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:58:30.489 DEBUG:teuthology.orchestra.run.smithi194:> uname -r 2024-09-05T00:58:30.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:58:30.490 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-09-05T00:58:30.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:58:30.491 DEBUG:teuthology.orchestra.run.smithi129:> uname -r 2024-09-05T00:58:30.546 INFO:teuthology.orchestra.run.smithi194.stdout:5.14.0-503.el9.x86_64 2024-09-05T00:58:30.546 INFO:teuthology.task.kernel:Running kernel on smithi194: 5.14.0-503.el9.x86_64 2024-09-05T00:58:30.547 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum install -y kernel 2024-09-05T00:58:30.548 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-503.el9.x86_64 2024-09-05T00:58:30.548 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-503.el9.x86_64 2024-09-05T00:58:30.549 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2024-09-05T00:58:30.550 INFO:teuthology.orchestra.run.smithi129.stdout:5.14.0-503.el9.x86_64 2024-09-05T00:58:30.550 INFO:teuthology.task.kernel:Running kernel on smithi129: 5.14.0-503.el9.x86_64 2024-09-05T00:58:30.550 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum install -y kernel 2024-09-05T00:58:32.926 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-05T00:58:33.792 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.2 MB 00:01 2024-09-05T00:58:34.253 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-05T00:58:37.356 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 20 MB 00:02 2024-09-05T00:58:37.677 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 20 MB 00:03 2024-09-05T00:58:38.271 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 20 MB 00:02 2024-09-05T00:58:42.494 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.5 MB 00:01 2024-09-05T00:58:42.729 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 9.9 MB/s | 6.5 MB 00:00 2024-09-05T00:58:44.169 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 19 kB 00:00 2024-09-05T00:58:44.555 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 19 kB 00:00 2024-09-05T00:58:45.090 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 6.5 MB 00:03 2024-09-05T00:58:45.357 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-05T00:58:45.718 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-05T00:58:46.632 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 19 kB 00:00 2024-09-05T00:58:47.428 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-05T00:58:50.763 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-05T00:58:51.124 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-05T00:58:52.702 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T00:58:52.703 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T00:58:52.782 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-05T00:58:52.783 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-09-05T00:58:52.784 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-05T00:58:52.901 DEBUG:teuthology.orchestra.run.smithi183:> echo no | sudo yum reinstall kernel || true 2024-09-05T00:58:52.908 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-09-05T00:58:53.130 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T00:58:53.130 INFO:teuthology.orchestra.run.smithi129.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T00:58:53.212 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-05T00:58:53.212 INFO:teuthology.orchestra.run.smithi129.stdout:Nothing to do. 2024-09-05T00:58:53.213 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-05T00:58:53.321 DEBUG:teuthology.orchestra.run.smithi129:> echo no | sudo yum reinstall kernel || true 2024-09-05T00:58:53.438 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 12:58:50 AM UTC. 2024-09-05T00:58:53.578 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-05T00:58:53.591 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:53.591 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:53.591 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:53.591 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 1.9 M 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-09-05T00:58:53.592 INFO:teuthology.orchestra.run.smithi183.stderr:Operation aborted. 2024-09-05T00:58:53.652 INFO:teuthology.orchestra.run.smithi183.stdout:Is this ok [y/N]: 2024-09-05T00:58:53.652 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum reinstall -y kernel || true 2024-09-05T00:58:53.862 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:02 ago on Thu 05 Sep 2024 12:58:51 AM UTC. 2024-09-05T00:58:54.006 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 1.9 M 2024-09-05T00:58:54.015 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-09-05T00:58:54.016 INFO:teuthology.orchestra.run.smithi129.stderr:Operation aborted. 2024-09-05T00:58:54.062 INFO:teuthology.orchestra.run.smithi129.stdout:Is this ok [y/N]: 2024-09-05T00:58:54.062 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum reinstall -y kernel || true 2024-09-05T00:58:54.185 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 12:58:50 AM UTC. 2024-09-05T00:58:54.312 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-05T00:58:54.313 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-09-05T00:58:54.314 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:54.314 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 1.9 M 2024-09-05T00:58:54.314 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-09-05T00:58:54.314 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-05T00:58:54.600 INFO:teuthology.orchestra.run.smithi129.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 12:58:51 AM UTC. 2024-09-05T00:58:54.726 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-05T00:58:54.727 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalling: 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:54.728 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-05T00:58:54.729 INFO:teuthology.orchestra.run.smithi129.stdout:================================================================================ 2024-09-05T00:58:54.729 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:54.729 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 1.9 M 2024-09-05T00:58:54.729 INFO:teuthology.orchestra.run.smithi129.stdout:Installed size: 0 2024-09-05T00:58:54.729 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-05T00:58:54.823 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-05T00:58:54.823 INFO:teuthology.orchestra.run.smithi194.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-05T00:58:54.904 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-09-05T00:58:54.905 INFO:teuthology.orchestra.run.smithi194.stdout:Nothing to do. 2024-09-05T00:58:54.905 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-09-05T00:58:55.001 DEBUG:teuthology.orchestra.run.smithi194:> echo no | sudo yum reinstall kernel || true 2024-09-05T00:58:55.181 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-503.el9.x86_64.rpm 2.4 MB/s | 1.9 MB 00:00 2024-09-05T00:58:55.182 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-05T00:58:55.182 INFO:teuthology.orchestra.run.smithi183.stdout:Total 2.2 MB/s | 1.9 MB 00:00 2024-09-05T00:58:55.212 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-05T00:58:55.466 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-05T00:58:55.466 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-05T00:58:55.496 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-05T00:58:55.496 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-05T00:58:55.543 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:03 ago on Thu 05 Sep 2024 12:58:52 AM UTC. 2024-09-05T00:58:55.651 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-05T00:58:55.678 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-09-05T00:58:55.686 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:Reinstalling: 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 1.9 M 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 0 2024-09-05T00:58:55.687 INFO:teuthology.orchestra.run.smithi194.stderr:Operation aborted. 2024-09-05T00:58:55.742 INFO:teuthology.orchestra.run.smithi183.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:55.742 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:55.746 INFO:teuthology.orchestra.run.smithi194.stdout:Is this ok [y/N]: 2024-09-05T00:58:55.746 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum reinstall -y kernel || true 2024-09-05T00:58:55.859 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-503.el9.x86_64.rpm 1.9 MB/s | 1.9 MB 00:00 2024-09-05T00:58:55.859 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-05T00:58:55.859 INFO:teuthology.orchestra.run.smithi129.stdout:Total 1.7 MB/s | 1.9 MB 00:01 2024-09-05T00:58:55.890 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-05T00:58:56.114 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-05T00:58:56.114 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-05T00:58:56.145 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-05T00:58:56.145 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-05T00:58:56.282 INFO:teuthology.orchestra.run.smithi194.stdout:Last metadata expiration check: 0:00:04 ago on Thu 05 Sep 2024 12:58:52 AM UTC. 2024-09-05T00:58:56.328 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-05T00:58:56.408 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-09-05T00:58:56.409 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:56.409 INFO:teuthology.orchestra.run.smithi194.stdout: Package Architecture Version Repository Size 2024-09-05T00:58:56.409 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:56.409 INFO:teuthology.orchestra.run.smithi194.stdout:Reinstalling: 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout: kernel x86_64 5.14.0-503.el9 baseos 1.9 M 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:================================================================================ 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 1.9 M 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:Installed size: 0 2024-09-05T00:58:56.410 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-09-05T00:58:56.419 INFO:teuthology.orchestra.run.smithi129.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:56.419 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:57.141 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:57.141 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalled: 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T00:58:57.323 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-05T00:58:57.538 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T00:58:57.596 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T00:58:57.597 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T00:58:57.597 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T00:58:57.597 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-05T00:58:57.597 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T00:58:57.619 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-5.14.0-503.el9.x86_64.rpm 1.6 MB/s | 1.9 MB 00:01 2024-09-05T00:58:57.619 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-09-05T00:58:57.619 INFO:teuthology.orchestra.run.smithi194.stdout:Total 1.5 MB/s | 1.9 MB 00:01 2024-09-05T00:58:57.626 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-09-05T00:58:57.649 DEBUG:teuthology.parallel:result is None 2024-09-05T00:58:57.656 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-09-05T00:58:57.844 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:57.845 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:57.845 INFO:teuthology.orchestra.run.smithi129.stdout:Reinstalled: 2024-09-05T00:58:57.845 INFO:teuthology.orchestra.run.smithi129.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T00:58:57.845 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T00:58:57.845 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-05T00:58:57.899 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-09-05T00:58:57.899 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-09-05T00:58:57.929 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-09-05T00:58:57.929 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-09-05T00:58:58.015 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T00:58:58.073 INFO:teuthology.orchestra.run.smithi129.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T00:58:58.073 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T00:58:58.074 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T00:58:58.074 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-05T00:58:58.074 DEBUG:teuthology.orchestra.run.smithi129:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T00:58:58.100 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-09-05T00:58:58.105 INFO:teuthology.orchestra.run.smithi129.stdout:ttyS1 2024-09-05T00:58:58.131 DEBUG:teuthology.parallel:result is None 2024-09-05T00:58:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: Reinstalling : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:58.204 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:58.937 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:58.937 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 1/2 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : kernel-5.14.0-503.el9.x86_64 2/2 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout:Reinstalled: 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout: kernel-5.14.0-503.el9.x86_64 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T00:58:59.706 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-09-05T00:58:59.908 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q kernel | sort -rV | head -n 1 2024-09-05T00:59:00.008 INFO:teuthology.orchestra.run.smithi194.stdout:kernel-5.14.0-503.el9.x86_64 2024-09-05T00:59:00.008 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-503.el9.x86_64 2024-09-05T00:59:00.008 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-09-05T00:59:00.009 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-05T00:59:00.009 DEBUG:teuthology.orchestra.run.smithi194:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-05T00:59:00.038 INFO:teuthology.orchestra.run.smithi194.stdout:ttyS1 2024-09-05T00:59:00.060 DEBUG:teuthology.parallel:result is None 2024-09-05T00:59:00.060 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T00:59:00.067 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T00:59:00.067 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T00:59:00.070 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T00:59:00.072 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T00:59:00.115 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T00:59:00.122 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T00:59:00.129 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T00:59:00.130 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T00:59:00.132 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T00:59:00.134 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T00:59:00.188 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T00:59:00.194 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T00:59:00.194 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T00:59:00.200 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T00:59:00.205 DEBUG:teuthology.orchestra.run.smithi194:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T00:59:00.261 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.263 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.272 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.275 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.289 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.299 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T00:59:00.301 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T00:59:00.308 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T00:59:00.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T00:59:00.311 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T00:59:00.313 DEBUG:teuthology.orchestra.run.smithi194:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T00:59:00.338 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T00:59:00.346 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T00:59:00.347 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T00:59:00.380 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T00:59:00.382 DEBUG:teuthology.orchestra.run.smithi194:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T00:59:00.397 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:00.491 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:00.532 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:00.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:00.660 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T00:59:00.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T00:59:00.728 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:00.772 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:00.821 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:00.903 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:00.947 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T00:59:00.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T00:59:01.013 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:01.058 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T00:59:01.106 DEBUG:teuthology.orchestra.run.smithi194:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:01.189 DEBUG:teuthology.orchestra.run.smithi194:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T00:59:01.230 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T00:59:01.230 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T00:59:01.295 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-05T00:59:01.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-09-05T00:59:01.300 DEBUG:teuthology.orchestra.run.smithi194:> sudo service rsyslog restart 2024-09-05T00:59:01.339 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T00:59:01.346 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T00:59:01.378 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T00:59:01.792 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T00:59:01.803 INFO:teuthology.task.internal:Starting timer... 2024-09-05T00:59:01.803 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T00:59:01.831 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T00:59:01.840 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-05T00:59:01.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-05T00:59:01.898 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-05T00:59:01.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-09-05T00:59:01.954 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-09-05T00:59:01.956 DEBUG:teuthology.orchestra.run.smithi194:> sudo service auditd rotate 2024-09-05T00:59:02.018 INFO:teuthology.orchestra.run.smithi194.stdout:Rotating logs: 2024-09-05T00:59:02.020 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T00:59:02.020 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-05T00:59:02.041 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-05T00:59:02.042 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-09-05T00:59:02.061 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-09-05T00:59:02.061 DEBUG:teuthology.orchestra.run.smithi194:> /usr/sbin/getenforce 2024-09-05T00:59:02.101 INFO:teuthology.orchestra.run.smithi194.stdout:Permissive 2024-09-05T00:59:02.102 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi129.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi194.front.sepia.ceph.com': 'permissive'} 2024-09-05T00:59:02.102 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T00:59:02.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:59:02.129 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T00:59:02.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:59:02.157 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T00:59:02.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T00:59:02.182 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T00:59:02.183 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-05T00:59:02.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-09-05T00:59:02.242 DEBUG:teuthology.orchestra.run.smithi194:> sudo /usr/sbin/setenforce permissive 2024-09-05T00:59:02.278 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T00:59:02.288 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-05T00:59:02.291 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-05T00:59:02.401 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T00:59:02.412 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T00:59:02.413 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi129.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi194.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T01:02:39.560 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi129.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi194.front.sepia.ceph.com')] 2024-09-05T01:02:39.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-05T01:02:39.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:02:39.642 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-05T01:02:39.724 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-05T01:02:39.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-05T01:02:39.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:02:39.799 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-09-05T01:02:39.839 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-09-05T01:02:39.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-05T01:02:39.840 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi194.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T01:02:39.914 DEBUG:teuthology.orchestra.run.smithi194:> true 2024-09-05T01:02:39.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi194.front.sepia.ceph.com' 2024-09-05T01:02:39.988 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T01:02:40.001 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T01:02:40.002 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:02:40.002 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:02:40.005 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:02:40.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:02:40.007 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T01:02:40.007 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:02:40.032 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:02:40.034 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:02:40.046 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:02:40.049 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:02:40.058 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T01:02:40.070 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-05T01:02:40.073 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-09-05T01:02:40.073 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T01:02:40.081 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-05T01:02:40.084 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-09-05T01:02:40.095 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:02:40.097 INFO:teuthology.orchestra.run.smithi194.stderr:sudo: ntpd: command not found 2024-09-05T01:02:40.098 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:02:40.109 INFO:teuthology.orchestra.run.smithi194.stdout:506 Cannot talk to daemon 2024-09-05T01:02:40.109 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:02:40.112 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:02:40.123 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T01:02:40.136 INFO:teuthology.orchestra.run.smithi194.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T01:02:40.171 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:02:40.172 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.175 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:02:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-05T01:02:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.176 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.189 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.193 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:02:40.193 INFO:teuthology.run_tasks:Running task install... 2024-09-05T01:02:40.201 DEBUG:teuthology.task.install:project ceph 2024-09-05T01:02:40.201 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'}} 2024-09-05T01:02:40.202 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4'} 2024-09-05T01:02:40.202 INFO:teuthology.task.install:Using flavor: default 2024-09-05T01:02:40.207 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T01:02:40.207 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-05T01:02:40.207 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:02:40.208 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:02:40.211 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:02:40.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:02:40.212 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'tag': None, 'wait_for_package': False} 2024-09-05T01:02:40.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:02:40.350 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:02:40.350 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:02:40.359 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:02:40.360 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:02:40.363 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/ 2024-09-05T01:02:40.363 INFO:teuthology.task.install.rpm:Package version is 19.1.1-259.g91c8a7e5 2024-09-05T01:02:40.462 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:02:40.463 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:02:40.463 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:02:40.468 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:02:40.468 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:02:40.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:02:40.474 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T01:02:40.474 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:02:40.474 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T01:02:40.489 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:02:40.489 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:02:40.494 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:02:40.494 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:02:40.500 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T01:02:40.501 DEBUG:teuthology.orchestra.run.smithi194:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T01:02:40.555 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:02:40.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:02:40.565 DEBUG:teuthology.orchestra.run.smithi194:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T01:02:40.630 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:02:40.636 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:02:40.640 DEBUG:teuthology.orchestra.run.smithi194:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T01:02:40.694 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-05T01:02:40.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-05T01:02:40.700 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-09-05T01:02:40.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-09-05T01:02:40.704 INFO:teuthology.orchestra.run.smithi194.stdout:check_obsoletes = 1 2024-09-05T01:02:40.706 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum clean all 2024-09-05T01:02:40.923 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-05T01:02:40.927 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-09-05T01:02:40.928 INFO:teuthology.orchestra.run.smithi194.stdout:45 files removed 2024-09-05T01:02:40.950 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:02:40.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:02:40.955 DEBUG:teuthology.orchestra.run.smithi194:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T01:02:41.488 INFO:teuthology.orchestra.run.smithi194.stdout:ceph packages for x86_64 319 kB/s | 84 kB 00:00 2024-09-05T01:02:41.491 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-05T01:02:41.523 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 280 kB/s | 84 kB 00:00 2024-09-05T01:02:41.773 INFO:teuthology.orchestra.run.smithi194.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-05T01:02:41.775 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-05T01:02:41.807 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-05T01:02:42.048 INFO:teuthology.orchestra.run.smithi194.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T01:02:42.051 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T01:02:42.083 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T01:02:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-05T01:02:44.232 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.2 MB 00:02 2024-09-05T01:02:45.859 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-05T01:02:46.517 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 20 MB 00:02 2024-09-05T01:02:48.302 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 15 MB/s | 20 MB 00:01 2024-09-05T01:02:49.641 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 20 MB 00:04 2024-09-05T01:02:51.069 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-05T01:02:52.561 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-05T01:02:53.340 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-05T01:02:54.008 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-05T01:02:54.294 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-05T01:02:55.484 INFO:teuthology.orchestra.run.smithi194.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-05T01:02:55.944 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-05T01:02:56.288 INFO:teuthology.orchestra.run.smithi194.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-05T01:02:56.841 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-05T01:02:59.680 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 2.0 kB/s | 1.7 kB 00:00 2024-09-05T01:03:01.802 INFO:teuthology.orchestra.run.smithi194.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-05T01:03:01.923 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:01.924 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:01.935 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:03:01.936 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:03:01.936 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:03:01.938 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:03:01.939 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:03:01.942 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:03:01.943 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:03:02.003 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-05T01:03:02.011 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-05T01:03:02.011 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:03:02.012 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:03:02.013 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:03:02.014 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:03:02.015 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:03:02.016 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:03:02.017 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:03:02.018 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:03:02.019 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:03:02.020 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:03:02.021 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:03:02.022 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:03:02.023 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:03:02.024 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 6 Packages 2024-09-05T01:03:02.025 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:02.026 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 222 M 2024-09-05T01:03:02.026 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-05T01:03:02.280 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-05T01:03:02.481 INFO:teuthology.orchestra.run.smithi129.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-05T01:03:02.656 INFO:teuthology.orchestra.run.smithi129.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 5.0 MB/s | 896 kB 00:00 2024-09-05T01:03:02.748 INFO:teuthology.orchestra.run.smithi129.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 1.6 MB/s | 145 kB 00:00 2024-09-05T01:03:02.857 INFO:teuthology.orchestra.run.smithi129.stdout:(4/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-05T01:03:02.932 INFO:teuthology.orchestra.run.smithi129.stdout:(5/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 13 MB/s | 2.4 MB 00:00 2024-09-05T01:03:02.981 INFO:teuthology.orchestra.run.smithi129.stdout:(6/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 12 MB/s | 1.5 MB 00:00 2024-09-05T01:03:03.325 INFO:teuthology.orchestra.run.smithi129.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-05T01:03:03.550 INFO:teuthology.orchestra.run.smithi129.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 18 MB/s | 21 MB 00:01 2024-09-05T01:03:03.625 INFO:teuthology.orchestra.run.smithi129.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 335 kB/s | 25 kB 00:00 2024-09-05T01:03:04.036 INFO:teuthology.orchestra.run.smithi194.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.037 INFO:teuthology.orchestra.run.smithi194.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.048 INFO:teuthology.orchestra.run.smithi194.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:03:04.049 INFO:teuthology.orchestra.run.smithi194.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.049 INFO:teuthology.orchestra.run.smithi194.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:03:04.050 INFO:teuthology.orchestra.run.smithi194.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.052 INFO:teuthology.orchestra.run.smithi194.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:03:04.054 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.055 INFO:teuthology.orchestra.run.smithi194.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.092 INFO:teuthology.orchestra.run.smithi129.stdout:(10/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 15 MB/s | 17 MB 00:01 2024-09-05T01:03:04.116 INFO:teuthology.orchestra.run.smithi194.stdout:Dependencies resolved. 2024-09-05T01:03:04.124 INFO:teuthology.orchestra.run.smithi194.stdout:====================================================================================== 2024-09-05T01:03:04.124 INFO:teuthology.orchestra.run.smithi194.stdout: Package Arch Version Repository Size 2024-09-05T01:03:04.124 INFO:teuthology.orchestra.run.smithi194.stdout:====================================================================================== 2024-09-05T01:03:04.124 INFO:teuthology.orchestra.run.smithi194.stdout:Installing: 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:03:04.125 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:03:04.126 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:03:04.127 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrading: 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout:Installing dependencies: 2024-09-05T01:03:04.128 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:03:04.129 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:03:04.130 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:03:04.131 INFO:teuthology.orchestra.run.smithi194.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:03:04.132 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:03:04.133 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:03:04.134 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:03:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:03:04.136 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:03:04.137 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:03:04.138 INFO:teuthology.orchestra.run.smithi194.stdout:Installing weak dependencies: 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction Summary 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout:====================================================================================== 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout:Install 148 Packages 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout:Upgrade 6 Packages 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:04.139 INFO:teuthology.orchestra.run.smithi194.stdout:Total download size: 222 M 2024-09-05T01:03:04.140 INFO:teuthology.orchestra.run.smithi194.stdout:Downloading Packages: 2024-09-05T01:03:04.209 INFO:teuthology.orchestra.run.smithi129.stdout:(11/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 12 MB/s | 11 MB 00:00 2024-09-05T01:03:04.243 INFO:teuthology.orchestra.run.smithi129.stdout:(12/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 1.5 MB/s | 224 kB 00:00 2024-09-05T01:03:04.268 INFO:teuthology.orchestra.run.smithi129.stdout:(13/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 461 kB/s | 27 kB 00:00 2024-09-05T01:03:04.293 INFO:teuthology.orchestra.run.smithi129.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 632 kB/s | 32 kB 00:00 2024-09-05T01:03:04.344 INFO:teuthology.orchestra.run.smithi129.stdout:(15/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-05T01:03:04.369 INFO:teuthology.orchestra.run.smithi129.stdout:(16/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 2.1 MB/s | 163 kB 00:00 2024-09-05T01:03:04.403 INFO:teuthology.orchestra.run.smithi129.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 2.1 MB/s | 127 kB 00:00 2024-09-05T01:03:04.436 INFO:teuthology.orchestra.run.smithi129.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 7.4 MB/s | 506 kB 00:00 2024-09-05T01:03:04.495 INFO:teuthology.orchestra.run.smithi129.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 771 kB/s | 45 kB 00:00 2024-09-05T01:03:04.562 INFO:teuthology.orchestra.run.smithi129.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 2.0 MB/s | 136 kB 00:00 2024-09-05T01:03:04.576 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.577 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.590 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T01:03:04.591 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T01:03:04.592 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T01:03:04.593 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.595 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T01:03:04.598 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.599 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T01:03:04.637 INFO:teuthology.orchestra.run.smithi129.stdout:(21/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 2.1 MB/s | 162 kB 00:00 2024-09-05T01:03:04.661 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 6.6 k 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.4 M 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 896 k 2024-09-05T01:03:04.672 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 145 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 1.5 M 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 146 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 3.6 M 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 7.4 M 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 49 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 11 M 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 49 M 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 285 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 765 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 224 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 27 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 18 k 2024-09-05T01:03:04.673 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 32 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 752 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 127 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T01:03:04.674 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 162 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 321 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 301 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 99 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 85 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.1 M 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 170 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T01:03:04.675 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.4 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 3.2 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 21 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 32 k 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 2.4 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 249 k 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 4.7 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 17 M 2024-09-05T01:03:04.676 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.1.1-259.g91c8a7e5.el9 ceph-noarch 16 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 25 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 163 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T01:03:04.677 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 506 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 5.3 M 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T01:03:04.678 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 45 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.1.1-259.g91c8a7e5.el9 ceph 136 k 2024-09-05T01:03:04.679 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T01:03:04.680 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T01:03:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T01:03:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T01:03:04.683 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T01:03:04.684 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout:====================================================================================== 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 6 Packages 2024-09-05T01:03:04.685 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:04.686 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 222 M 2024-09-05T01:03:04.686 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-09-05T01:03:04.738 INFO:teuthology.orchestra.run.smithi129.stdout:(22/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-05T01:03:04.763 INFO:teuthology.orchestra.run.smithi129.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 2.5 MB/s | 321 kB 00:00 2024-09-05T01:03:04.797 INFO:teuthology.orchestra.run.smithi129.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 5.0 MB/s | 301 kB 00:00 2024-09-05T01:03:04.822 INFO:teuthology.orchestra.run.smithi129.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T01:03:04.855 INFO:teuthology.orchestra.run.smithi129.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-05T01:03:04.923 INFO:teuthology.orchestra.run.smithi129.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-05T01:03:04.990 INFO:teuthology.orchestra.run.smithi129.stdout:(28/154): ceph-grafana-dashboards-19.1.1-259.g9 478 kB/s | 32 kB 00:00 2024-09-05T01:03:05.048 INFO:teuthology.orchestra.run.smithi129.stdout:(29/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-05T01:03:05.069 INFO:teuthology.orchestra.run.smithi194.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-05T01:03:05.074 INFO:teuthology.orchestra.run.smithi129.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 1.7 MB/s | 146 kB 00:00 2024-09-05T01:03:05.106 INFO:teuthology.orchestra.run.smithi183.stdout:(1/154): ceph-19.1.1-259.g91c8a7e5.el9.x86_64.r 42 kB/s | 6.6 kB 00:00 2024-09-05T01:03:05.366 INFO:teuthology.orchestra.run.smithi129.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 11 MB/s | 3.6 MB 00:00 2024-09-05T01:03:05.449 INFO:teuthology.orchestra.run.smithi129.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 2.9 MB/s | 249 kB 00:00 2024-09-05T01:03:05.525 INFO:teuthology.orchestra.run.smithi129.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 658 kB/s | 49 kB 00:00 2024-09-05T01:03:05.598 INFO:teuthology.orchestra.run.smithi183.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.8 MB/s | 896 kB 00:00 2024-09-05T01:03:05.636 INFO:teuthology.orchestra.run.smithi194.stdout:(2/154): ceph-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.5 MB/s | 896 kB 00:00 2024-09-05T01:03:05.658 INFO:teuthology.orchestra.run.smithi129.stdout:(34/154): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-05T01:03:05.665 INFO:teuthology.orchestra.run.smithi183.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 2.1 MB/s | 145 kB 00:00 2024-09-05T01:03:05.675 INFO:teuthology.orchestra.run.smithi129.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 109 kB/s | 16 kB 00:00 2024-09-05T01:03:05.703 INFO:teuthology.orchestra.run.smithi194.stdout:(3/154): ceph-immutable-object-cache-19.1.1-259 2.1 MB/s | 145 kB 00:00 2024-09-05T01:03:06.007 INFO:teuthology.orchestra.run.smithi183.stdout:(4/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 7.0 MB/s | 2.4 MB 00:00 2024-09-05T01:03:06.079 INFO:teuthology.orchestra.run.smithi194.stdout:(4/154): ceph-mds-19.1.1-259.g91c8a7e5.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-05T01:03:06.101 INFO:teuthology.orchestra.run.smithi129.stdout:(36/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 20 MB/s | 49 MB 00:02 2024-09-05T01:03:06.126 INFO:teuthology.orchestra.run.smithi129.stdout:(37/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 610 kB/s | 285 kB 00:00 2024-09-05T01:03:06.152 INFO:teuthology.orchestra.run.smithi129.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 365 kB/s | 18 kB 00:00 2024-09-05T01:03:06.173 INFO:teuthology.orchestra.run.smithi183.stdout:(5/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 4.4 MB/s | 5.4 MB 00:01 2024-09-05T01:03:06.187 INFO:teuthology.orchestra.run.smithi194.stdout:(5/154): ceph-base-19.1.1-259.g91c8a7e5.el9.x86 4.3 MB/s | 5.4 MB 00:01 2024-09-05T01:03:06.194 INFO:teuthology.orchestra.run.smithi129.stdout:(39/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 1.4 MB/s | 765 kB 00:00 2024-09-05T01:03:06.232 INFO:teuthology.orchestra.run.smithi183.stdout:(6/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 6.8 MB/s | 1.5 MB 00:00 2024-09-05T01:03:06.261 INFO:teuthology.orchestra.run.smithi129.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 596 kB/s | 80 kB 00:00 2024-09-05T01:03:06.312 INFO:teuthology.orchestra.run.smithi194.stdout:(6/154): ceph-mgr-19.1.1-259.g91c8a7e5.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-05T01:03:06.336 INFO:teuthology.orchestra.run.smithi129.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-05T01:03:06.362 INFO:teuthology.orchestra.run.smithi129.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-05T01:03:06.386 INFO:teuthology.orchestra.run.smithi129.stdout:(43/154): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-05T01:03:06.412 INFO:teuthology.orchestra.run.smithi129.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-05T01:03:06.454 INFO:teuthology.orchestra.run.smithi129.stdout:(45/154): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-09-05T01:03:06.479 INFO:teuthology.orchestra.run.smithi129.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-05T01:03:06.504 INFO:teuthology.orchestra.run.smithi129.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-05T01:03:06.638 INFO:teuthology.orchestra.run.smithi129.stdout:(48/154): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-09-05T01:03:06.757 INFO:teuthology.orchestra.run.smithi183.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 8.1 MB/s | 4.7 MB 00:00 2024-09-05T01:03:06.788 INFO:teuthology.orchestra.run.smithi194.stdout:(7/154): ceph-mon-19.1.1-259.g91c8a7e5.el9.x86_ 7.8 MB/s | 4.7 MB 00:00 2024-09-05T01:03:07.055 INFO:teuthology.orchestra.run.smithi129.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 256 kB/s | 107 kB 00:00 2024-09-05T01:03:07.164 INFO:teuthology.orchestra.run.smithi129.stdout:(50/154): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-05T01:03:07.189 INFO:teuthology.orchestra.run.smithi129.stdout:(51/154): automake-1.16.2-8.el9.noarch.rpm 971 kB/s | 689 kB 00:00 2024-09-05T01:03:07.289 INFO:teuthology.orchestra.run.smithi129.stdout:(52/154): bison-3.7.4-5.el9.x86_64.rpm 1.2 MB/s | 943 kB 00:00 2024-09-05T01:03:07.314 INFO:teuthology.orchestra.run.smithi129.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 260 kB/s | 33 kB 00:00 2024-09-05T01:03:07.415 INFO:teuthology.orchestra.run.smithi129.stdout:(54/154): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-05T01:03:07.421 INFO:teuthology.orchestra.run.smithi194.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-05T01:03:07.440 INFO:teuthology.orchestra.run.smithi129.stdout:(55/154): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-09-05T01:03:07.457 INFO:teuthology.orchestra.run.smithi183.stdout:(8/154): ceph-common-19.1.1-259.g91c8a7e5.el9.x 8.5 MB/s | 21 MB 00:02 2024-09-05T01:03:07.480 INFO:teuthology.orchestra.run.smithi194.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 434 kB/s | 25 kB 00:00 2024-09-05T01:03:07.516 INFO:teuthology.orchestra.run.smithi129.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-05T01:03:07.533 INFO:teuthology.orchestra.run.smithi183.stdout:(9/154): ceph-selinux-19.1.1-259.g91c8a7e5.el9. 336 kB/s | 25 kB 00:00 2024-09-05T01:03:07.549 INFO:teuthology.orchestra.run.smithi129.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-05T01:03:07.658 INFO:teuthology.orchestra.run.smithi129.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-09-05T01:03:07.772 INFO:teuthology.orchestra.run.smithi194.stdout:(10/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 11 MB/s | 11 MB 00:00 2024-09-05T01:03:07.792 INFO:teuthology.orchestra.run.smithi129.stdout:(59/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-09-05T01:03:07.817 INFO:teuthology.orchestra.run.smithi129.stdout:(60/154): librdkafka-1.6.1-102.el9.x86_64.rpm 2.4 MB/s | 662 kB 00:00 2024-09-05T01:03:07.863 INFO:teuthology.orchestra.run.smithi194.stdout:(11/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 2.4 MB/s | 224 kB 00:00 2024-09-05T01:03:07.892 INFO:teuthology.orchestra.run.smithi129.stdout:(61/154): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-09-05T01:03:07.917 INFO:teuthology.orchestra.run.smithi129.stdout:(62/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 195 kB/s | 24 kB 00:00 2024-09-05T01:03:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 377 kB/s | 47 kB 00:00 2024-09-05T01:03:07.955 INFO:teuthology.orchestra.run.smithi194.stdout:(12/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 294 kB/s | 27 kB 00:00 2024-09-05T01:03:08.002 INFO:teuthology.orchestra.run.smithi129.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-05T01:03:08.044 INFO:teuthology.orchestra.run.smithi129.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-05T01:03:08.066 INFO:teuthology.orchestra.run.smithi183.stdout:(10/154): ceph-radosgw-19.1.1-259.g91c8a7e5.el9 8.1 MB/s | 11 MB 00:01 2024-09-05T01:03:08.139 INFO:teuthology.orchestra.run.smithi194.stdout:(13/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 9.2 MB/s | 17 MB 00:01 2024-09-05T01:03:08.164 INFO:teuthology.orchestra.run.smithi194.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 152 kB/s | 32 kB 00:00 2024-09-05T01:03:08.186 INFO:teuthology.orchestra.run.smithi129.stdout:(66/154): lua-5.4.4-4.el9.x86_64.rpm 704 kB/s | 188 kB 00:00 2024-09-05T01:03:08.215 INFO:teuthology.orchestra.run.smithi194.stdout:(15/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 9.8 MB/s | 752 kB 00:00 2024-09-05T01:03:08.233 INFO:teuthology.orchestra.run.smithi183.stdout:(11/154): ceph-osd-19.1.1-259.g91c8a7e5.el9.x86 8.4 MB/s | 17 MB 00:02 2024-09-05T01:03:08.240 INFO:teuthology.orchestra.run.smithi194.stdout:(16/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 2.1 MB/s | 163 kB 00:00 2024-09-05T01:03:08.265 INFO:teuthology.orchestra.run.smithi194.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 2.5 MB/s | 127 kB 00:00 2024-09-05T01:03:08.267 INFO:teuthology.orchestra.run.smithi183.stdout:(12/154): cephfs-mirror-19.1.1-259.g91c8a7e5.el 1.1 MB/s | 224 kB 00:00 2024-09-05T01:03:08.292 INFO:teuthology.orchestra.run.smithi183.stdout:(13/154): cephfs-shell-19.1.1-259.g91c8a7e5.el9 463 kB/s | 27 kB 00:00 2024-09-05T01:03:08.316 INFO:teuthology.orchestra.run.smithi194.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 6.6 MB/s | 506 kB 00:00 2024-09-05T01:03:08.334 INFO:teuthology.orchestra.run.smithi183.stdout:(14/154): libcephfs-devel-19.1.1-259.g91c8a7e5. 475 kB/s | 32 kB 00:00 2024-09-05T01:03:08.399 INFO:teuthology.orchestra.run.smithi194.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 541 kB/s | 45 kB 00:00 2024-09-05T01:03:08.418 INFO:teuthology.orchestra.run.smithi183.stdout:(15/154): libcephfs2-19.1.1-259.g91c8a7e5.el9.x 5.9 MB/s | 752 kB 00:00 2024-09-05T01:03:08.443 INFO:teuthology.orchestra.run.smithi183.stdout:(16/154): libcephsqlite-19.1.1-259.g91c8a7e5.el 1.5 MB/s | 163 kB 00:00 2024-09-05T01:03:08.474 INFO:teuthology.orchestra.run.smithi194.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 1.8 MB/s | 136 kB 00:00 2024-09-05T01:03:08.510 INFO:teuthology.orchestra.run.smithi183.stdout:(17/154): librados-devel-19.1.1-259.g91c8a7e5.e 1.3 MB/s | 127 kB 00:00 2024-09-05T01:03:08.535 INFO:teuthology.orchestra.run.smithi183.stdout:(18/154): libradosstriper1-19.1.1-259.g91c8a7e5 5.4 MB/s | 506 kB 00:00 2024-09-05T01:03:08.594 INFO:teuthology.orchestra.run.smithi183.stdout:(19/154): python3-ceph-argparse-19.1.1-259.g91c 775 kB/s | 45 kB 00:00 2024-09-05T01:03:08.600 INFO:teuthology.orchestra.run.smithi194.stdout:(21/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-05T01:03:08.625 INFO:teuthology.orchestra.run.smithi194.stdout:(22/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 1.1 MB/s | 162 kB 00:00 2024-09-05T01:03:08.661 INFO:teuthology.orchestra.run.smithi183.stdout:(20/154): python3-ceph-common-19.1.1-259.g91c8a 2.0 MB/s | 136 kB 00:00 2024-09-05T01:03:08.675 INFO:teuthology.orchestra.run.smithi194.stdout:(23/154): python3-rados-19.1.1-259.g91c8a7e5.el 4.2 MB/s | 321 kB 00:00 2024-09-05T01:03:08.709 INFO:teuthology.orchestra.run.smithi194.stdout:(24/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 3.5 MB/s | 301 kB 00:00 2024-09-05T01:03:08.736 INFO:teuthology.orchestra.run.smithi183.stdout:(21/154): python3-cephfs-19.1.1-259.g91c8a7e5.e 2.1 MB/s | 162 kB 00:00 2024-09-05T01:03:08.751 INFO:teuthology.orchestra.run.smithi194.stdout:(25/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.3 MB/s | 99 kB 00:00 2024-09-05T01:03:08.776 INFO:teuthology.orchestra.run.smithi194.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-05T01:03:08.812 INFO:teuthology.orchestra.run.smithi183.stdout:(22/154): python3-rados-19.1.1-259.g91c8a7e5.el 4.2 MB/s | 321 kB 00:00 2024-09-05T01:03:08.876 INFO:teuthology.orchestra.run.smithi194.stdout:(27/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-05T01:03:08.920 INFO:teuthology.orchestra.run.smithi183.stdout:(23/154): python3-rbd-19.1.1-259.g91c8a7e5.el9. 2.7 MB/s | 301 kB 00:00 2024-09-05T01:03:08.993 INFO:teuthology.orchestra.run.smithi194.stdout:(28/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-05T01:03:08.995 INFO:teuthology.orchestra.run.smithi183.stdout:(24/154): python3-rgw-19.1.1-259.g91c8a7e5.el9. 1.3 MB/s | 99 kB 00:00 2024-09-05T01:03:09.019 INFO:teuthology.orchestra.run.smithi194.stdout:(29/154): ceph-grafana-dashboards-19.1.1-259.g9 224 kB/s | 32 kB 00:00 2024-09-05T01:03:09.044 INFO:teuthology.orchestra.run.smithi194.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.8 MB/s | 146 kB 00:00 2024-09-05T01:03:09.079 INFO:teuthology.orchestra.run.smithi183.stdout:(25/154): librgw2-19.1.1-259.g91c8a7e5.el9.x86_ 9.4 MB/s | 5.3 MB 00:00 2024-09-05T01:03:09.104 INFO:teuthology.orchestra.run.smithi183.stdout:(26/154): rbd-fuse-19.1.1-259.g91c8a7e5.el9.x86 786 kB/s | 85 kB 00:00 2024-09-05T01:03:09.361 INFO:teuthology.orchestra.run.smithi194.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 10 MB/s | 3.6 MB 00:00 2024-09-05T01:03:09.394 INFO:teuthology.orchestra.run.smithi129.stdout:(67/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 3.3 MB/s | 4.4 MB 00:01 2024-09-05T01:03:09.495 INFO:teuthology.orchestra.run.smithi194.stdout:(32/154): ceph-mgr-diskprediction-local-19.1.1- 16 MB/s | 7.4 MB 00:00 2024-09-05T01:03:09.524 INFO:teuthology.orchestra.run.smithi194.stdout:(33/154): ceph-mgr-modules-core-19.1.1-259.g91c 1.5 MB/s | 249 kB 00:00 2024-09-05T01:03:09.554 INFO:teuthology.orchestra.run.smithi194.stdout:(34/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 846 kB/s | 49 kB 00:00 2024-09-05T01:03:09.555 INFO:teuthology.orchestra.run.smithi183.stdout:(27/154): rbd-mirror-19.1.1-259.g91c8a7e5.el9.x 6.5 MB/s | 3.1 MB 00:00 2024-09-05T01:03:09.579 INFO:teuthology.orchestra.run.smithi194.stdout:(35/154): ceph-prometheus-alerts-19.1.1-259.g91 300 kB/s | 16 kB 00:00 2024-09-05T01:03:09.612 INFO:teuthology.orchestra.run.smithi194.stdout:(36/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 4.8 MB/s | 285 kB 00:00 2024-09-05T01:03:09.613 INFO:teuthology.orchestra.run.smithi183.stdout:(28/154): ceph-grafana-dashboards-19.1.1-259.g9 545 kB/s | 32 kB 00:00 2024-09-05T01:03:09.646 INFO:teuthology.orchestra.run.smithi194.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 11 MB/s | 765 kB 00:00 2024-09-05T01:03:09.647 INFO:teuthology.orchestra.run.smithi183.stdout:(29/154): rbd-nbd-19.1.1-259.g91c8a7e5.el9.x86_ 314 kB/s | 170 kB 00:00 2024-09-05T01:03:09.671 INFO:teuthology.orchestra.run.smithi194.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 311 kB/s | 18 kB 00:00 2024-09-05T01:03:09.672 INFO:teuthology.orchestra.run.smithi183.stdout:(30/154): ceph-mgr-cephadm-19.1.1-259.g91c8a7e5 2.4 MB/s | 146 kB 00:00 2024-09-05T01:03:09.788 INFO:teuthology.orchestra.run.smithi194.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-05T01:03:09.863 INFO:teuthology.orchestra.run.smithi194.stdout:(40/154): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-05T01:03:09.889 INFO:teuthology.orchestra.run.smithi194.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-05T01:03:09.922 INFO:teuthology.orchestra.run.smithi194.stdout:(42/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-05T01:03:09.973 INFO:teuthology.orchestra.run.smithi194.stdout:(43/154): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-05T01:03:09.998 INFO:teuthology.orchestra.run.smithi194.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-05T01:03:10.023 INFO:teuthology.orchestra.run.smithi194.stdout:(45/154): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-05T01:03:10.031 INFO:teuthology.orchestra.run.smithi183.stdout:(31/154): ceph-mgr-dashboard-19.1.1-259.g91c8a7 9.3 MB/s | 3.6 MB 00:00 2024-09-05T01:03:10.048 INFO:teuthology.orchestra.run.smithi194.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-05T01:03:10.074 INFO:teuthology.orchestra.run.smithi194.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-05T01:03:10.089 INFO:teuthology.orchestra.run.smithi183.stdout:(32/154): ceph-mgr-modules-core-19.1.1-259.g91c 4.2 MB/s | 249 kB 00:00 2024-09-05T01:03:10.140 INFO:teuthology.orchestra.run.smithi183.stdout:(33/154): ceph-mgr-rook-19.1.1-259.g91c8a7e5.el 986 kB/s | 49 kB 00:00 2024-09-05T01:03:10.186 INFO:teuthology.orchestra.run.smithi129.stdout:(68/154): postgresql-13.16-1.el9.x86_64.rpm 812 kB/s | 1.6 MB 00:01 2024-09-05T01:03:10.190 INFO:teuthology.orchestra.run.smithi183.stdout:(34/154): ceph-prometheus-alerts-19.1.1-259.g91 328 kB/s | 16 kB 00:00 2024-09-05T01:03:10.191 INFO:teuthology.orchestra.run.smithi194.stdout:(48/154): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-09-05T01:03:10.257 INFO:teuthology.orchestra.run.smithi183.stdout:(35/154): ceph-volume-19.1.1-259.g91c8a7e5.el9. 4.2 MB/s | 285 kB 00:00 2024-09-05T01:03:10.257 INFO:teuthology.orchestra.run.smithi194.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-05T01:03:10.291 INFO:teuthology.orchestra.run.smithi194.stdout:(50/154): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-05T01:03:10.294 INFO:teuthology.orchestra.run.smithi129.stdout:(69/154): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-05T01:03:10.308 INFO:teuthology.orchestra.run.smithi194.stdout:(51/154): elfutils-libelf-devel-0.191-4.el9.x86 430 kB/s | 22 kB 00:00 2024-09-05T01:03:10.366 INFO:teuthology.orchestra.run.smithi183.stdout:(36/154): ceph-mgr-diskprediction-local-19.1.1- 11 MB/s | 7.4 MB 00:00 2024-09-05T01:03:10.409 INFO:teuthology.orchestra.run.smithi183.stdout:(37/154): cephadm-19.1.1-259.g91c8a7e5.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-05T01:03:10.433 INFO:teuthology.orchestra.run.smithi183.stdout:(38/154): cephfs-top-19.1.1-259.g91c8a7e5.el9.n 273 kB/s | 18 kB 00:00 2024-09-05T01:03:10.542 INFO:teuthology.orchestra.run.smithi183.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-05T01:03:10.545 INFO:teuthology.orchestra.run.smithi129.stdout:(70/154): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-05T01:03:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:(71/154): postgresql-contrib-13.16-1.el9.x86_64 717 kB/s | 910 kB 00:01 2024-09-05T01:03:10.704 INFO:teuthology.orchestra.run.smithi129.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 6.4 MB/s | 1.0 MB 00:00 2024-09-05T01:03:10.742 INFO:teuthology.orchestra.run.smithi194.stdout:(52/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 15 MB/s | 49 MB 00:03 2024-09-05T01:03:10.759 INFO:teuthology.orchestra.run.smithi194.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 73 kB/s | 33 kB 00:00 2024-09-05T01:03:10.792 INFO:teuthology.orchestra.run.smithi194.stdout:(54/154): flex-2.6.4-9.el9.x86_64.rpm 626 kB/s | 313 kB 00:00 2024-09-05T01:03:10.834 INFO:teuthology.orchestra.run.smithi194.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 242 kB/s | 18 kB 00:00 2024-09-05T01:03:10.854 INFO:teuthology.orchestra.run.smithi129.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 459 kB/s | 87 kB 00:00 2024-09-05T01:03:10.860 INFO:teuthology.orchestra.run.smithi194.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-05T01:03:10.910 INFO:teuthology.orchestra.run.smithi194.stdout:(57/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-09-05T01:03:10.935 INFO:teuthology.orchestra.run.smithi194.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-09-05T01:03:10.971 INFO:teuthology.orchestra.run.smithi129.stdout:(74/154): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-05T01:03:10.975 INFO:teuthology.orchestra.run.smithi183.stdout:(40/154): ceph-test-19.1.1-259.g91c8a7e5.el9.x8 14 MB/s | 49 MB 00:03 2024-09-05T01:03:10.996 INFO:teuthology.orchestra.run.smithi129.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 432 kB/s | 61 kB 00:00 2024-09-05T01:03:11.000 INFO:teuthology.orchestra.run.smithi183.stdout:(41/154): libquadmath-11.5.0-2.el9.x86_64.rpm 417 kB/s | 191 kB 00:00 2024-09-05T01:03:11.019 INFO:teuthology.orchestra.run.smithi194.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-05T01:03:11.034 INFO:teuthology.orchestra.run.smithi183.stdout:(42/154): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-05T01:03:11.052 INFO:teuthology.orchestra.run.smithi194.stdout:(60/154): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-05T01:03:11.076 INFO:teuthology.orchestra.run.smithi183.stdout:(43/154): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-05T01:03:11.080 INFO:teuthology.orchestra.run.smithi129.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-05T01:03:11.086 INFO:teuthology.orchestra.run.smithi194.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-05T01:03:11.118 INFO:teuthology.orchestra.run.smithi183.stdout:(44/154): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-09-05T01:03:11.128 INFO:teuthology.orchestra.run.smithi194.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-09-05T01:03:11.152 INFO:teuthology.orchestra.run.smithi183.stdout:(45/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-05T01:03:11.162 INFO:teuthology.orchestra.run.smithi194.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-05T01:03:11.177 INFO:teuthology.orchestra.run.smithi183.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-05T01:03:11.180 INFO:teuthology.orchestra.run.smithi129.stdout:(77/154): python3-jsonpatch-1.21-16.el9.noarch. 256 kB/s | 26 kB 00:00 2024-09-05T01:03:11.203 INFO:teuthology.orchestra.run.smithi194.stdout:(64/154): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-05T01:03:11.211 INFO:teuthology.orchestra.run.smithi183.stdout:(47/154): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-05T01:03:11.229 INFO:teuthology.orchestra.run.smithi194.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 577 kB/s | 38 kB 00:00 2024-09-05T01:03:11.236 INFO:teuthology.orchestra.run.smithi183.stdout:(48/154): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-05T01:03:11.247 INFO:teuthology.orchestra.run.smithi129.stdout:(78/154): openblas-openmp-0.3.26-2.el9.x86_64.r 1.5 MB/s | 4.9 MB 00:03 2024-09-05T01:03:11.279 INFO:teuthology.orchestra.run.smithi194.stdout:(66/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.6 MB/s | 3.0 MB 00:00 2024-09-05T01:03:11.286 INFO:teuthology.orchestra.run.smithi129.stdout:(79/154): python3-jsonpointer-2.0-4.el9.noarch. 182 kB/s | 19 kB 00:00 2024-09-05T01:03:11.353 INFO:teuthology.orchestra.run.smithi129.stdout:(80/154): python3-jinja2-2.11.3-6.el9.noarch.rp 696 kB/s | 248 kB 00:00 2024-09-05T01:03:11.373 INFO:teuthology.orchestra.run.smithi183.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 781 kB/s | 107 kB 00:00 2024-09-05T01:03:11.378 INFO:teuthology.orchestra.run.smithi129.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-05T01:03:11.396 INFO:teuthology.orchestra.run.smithi194.stdout:(67/154): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-05T01:03:11.403 INFO:teuthology.orchestra.run.smithi129.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-09-05T01:03:11.404 INFO:teuthology.orchestra.run.smithi183.stdout:(50/154): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-05T01:03:11.429 INFO:teuthology.orchestra.run.smithi183.stdout:(51/154): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-09-05T01:03:11.454 INFO:teuthology.orchestra.run.smithi183.stdout:(52/154): elfutils-libelf-devel-0.191-4.el9.x86 270 kB/s | 22 kB 00:00 2024-09-05T01:03:11.463 INFO:teuthology.orchestra.run.smithi194.stdout:(68/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-05T01:03:11.488 INFO:teuthology.orchestra.run.smithi183.stdout:(53/154): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-05T01:03:11.495 INFO:teuthology.orchestra.run.smithi129.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-05T01:03:11.505 INFO:teuthology.orchestra.run.smithi194.stdout:(69/154): postgresql-13.16-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-09-05T01:03:11.513 INFO:teuthology.orchestra.run.smithi183.stdout:(54/154): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-05T01:03:11.521 INFO:teuthology.orchestra.run.smithi129.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-05T01:03:11.530 INFO:teuthology.orchestra.run.smithi194.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-05T01:03:11.555 INFO:teuthology.orchestra.run.smithi183.stdout:(55/154): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-09-05T01:03:11.571 INFO:teuthology.orchestra.run.smithi194.stdout:(71/154): postgresql-contrib-13.16-1.el9.x86_64 5.1 MB/s | 910 kB 00:00 2024-09-05T01:03:11.597 INFO:teuthology.orchestra.run.smithi183.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-09-05T01:03:11.629 INFO:teuthology.orchestra.run.smithi194.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-05T01:03:11.630 INFO:teuthology.orchestra.run.smithi129.stdout:(85/154): python3-prettytable-0.7.2-27.el9.noar 390 kB/s | 42 kB 00:00 2024-09-05T01:03:11.654 INFO:teuthology.orchestra.run.smithi194.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-05T01:03:11.655 INFO:teuthology.orchestra.run.smithi183.stdout:(57/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-05T01:03:11.680 INFO:teuthology.orchestra.run.smithi129.stdout:(86/154): python3-packaging-20.9-5.el9.noarch.r 419 kB/s | 77 kB 00:00 2024-09-05T01:03:11.681 INFO:teuthology.orchestra.run.smithi183.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-09-05T01:03:11.706 INFO:teuthology.orchestra.run.smithi183.stdout:(59/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-05T01:03:11.721 INFO:teuthology.orchestra.run.smithi194.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 920 kB/s | 61 kB 00:00 2024-09-05T01:03:11.740 INFO:teuthology.orchestra.run.smithi183.stdout:(60/154): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-09-05T01:03:11.747 INFO:teuthology.orchestra.run.smithi129.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-05T01:03:11.765 INFO:teuthology.orchestra.run.smithi183.stdout:(61/154): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-05T01:03:11.790 INFO:teuthology.orchestra.run.smithi183.stdout:(62/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-05T01:03:11.817 INFO:teuthology.orchestra.run.smithi183.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 616 kB/s | 47 kB 00:00 2024-09-05T01:03:11.830 INFO:teuthology.orchestra.run.smithi194.stdout:(75/154): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-09-05T01:03:11.850 INFO:teuthology.orchestra.run.smithi183.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.3 MB/s | 292 kB 00:00 2024-09-05T01:03:11.855 INFO:teuthology.orchestra.run.smithi129.stdout:(88/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-05T01:03:11.856 INFO:teuthology.orchestra.run.smithi194.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-05T01:03:11.884 INFO:teuthology.orchestra.run.smithi183.stdout:(65/154): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-05T01:03:11.910 INFO:teuthology.orchestra.run.smithi183.stdout:(66/154): openblas-0.3.26-2.el9.x86_64.rpm 417 kB/s | 38 kB 00:00 2024-09-05T01:03:11.931 INFO:teuthology.orchestra.run.smithi194.stdout:(77/154): postgresql-server-13.16-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-09-05T01:03:11.955 INFO:teuthology.orchestra.run.smithi129.stdout:(89/154): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-09-05T01:03:11.956 INFO:teuthology.orchestra.run.smithi194.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-09-05T01:03:11.981 INFO:teuthology.orchestra.run.smithi129.stdout:(90/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 528 kB/s | 159 kB 00:00 2024-09-05T01:03:11.981 INFO:teuthology.orchestra.run.smithi194.stdout:(79/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-05T01:03:12.002 INFO:teuthology.orchestra.run.smithi183.stdout:(67/154): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-05T01:03:12.006 INFO:teuthology.orchestra.run.smithi194.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-09-05T01:03:12.032 INFO:teuthology.orchestra.run.smithi194.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-05T01:03:12.043 INFO:teuthology.orchestra.run.smithi183.stdout:(68/154): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-05T01:03:12.056 INFO:teuthology.orchestra.run.smithi129.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 535 kB/s | 54 kB 00:00 2024-09-05T01:03:12.057 INFO:teuthology.orchestra.run.smithi194.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-09-05T01:03:12.102 INFO:teuthology.orchestra.run.smithi183.stdout:(69/154): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-05T01:03:12.116 INFO:teuthology.orchestra.run.smithi194.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-05T01:03:12.127 INFO:teuthology.orchestra.run.smithi183.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-05T01:03:12.141 INFO:teuthology.orchestra.run.smithi194.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-05T01:03:12.148 INFO:teuthology.orchestra.run.smithi129.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 455 kB/s | 42 kB 00:00 2024-09-05T01:03:12.186 INFO:teuthology.orchestra.run.smithi183.stdout:(71/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-05T01:03:12.191 INFO:teuthology.orchestra.run.smithi194.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-09-05T01:03:12.216 INFO:teuthology.orchestra.run.smithi194.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-09-05T01:03:12.240 INFO:teuthology.orchestra.run.smithi129.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 481 kB/s | 44 kB 00:00 2024-09-05T01:03:12.245 INFO:teuthology.orchestra.run.smithi183.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-09-05T01:03:12.259 INFO:teuthology.orchestra.run.smithi194.stdout:(87/154): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-05T01:03:12.270 INFO:teuthology.orchestra.run.smithi183.stdout:(73/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-05T01:03:12.292 INFO:teuthology.orchestra.run.smithi194.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-05T01:03:12.345 INFO:teuthology.orchestra.run.smithi183.stdout:(74/154): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-05T01:03:12.349 INFO:teuthology.orchestra.run.smithi129.stdout:(94/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-05T01:03:12.362 INFO:teuthology.orchestra.run.smithi183.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 665 kB/s | 61 kB 00:00 2024-09-05T01:03:12.367 INFO:teuthology.orchestra.run.smithi194.stdout:(89/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-09-05T01:03:12.401 INFO:teuthology.orchestra.run.smithi194.stdout:(90/154): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 279 kB 00:00 2024-09-05T01:03:12.412 INFO:teuthology.orchestra.run.smithi183.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-05T01:03:12.426 INFO:teuthology.orchestra.run.smithi194.stdout:(91/154): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-09-05T01:03:12.441 INFO:teuthology.orchestra.run.smithi129.stdout:(95/154): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-09-05T01:03:12.446 INFO:teuthology.orchestra.run.smithi183.stdout:(77/154): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-05T01:03:12.451 INFO:teuthology.orchestra.run.smithi194.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 643 kB/s | 54 kB 00:00 2024-09-05T01:03:12.493 INFO:teuthology.orchestra.run.smithi194.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 626 kB/s | 42 kB 00:00 2024-09-05T01:03:12.518 INFO:teuthology.orchestra.run.smithi194.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-09-05T01:03:12.522 INFO:teuthology.orchestra.run.smithi183.stdout:(78/154): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-05T01:03:12.538 INFO:teuthology.orchestra.run.smithi183.stdout:(79/154): python3-jsonpatch-1.21-16.el9.noarch. 204 kB/s | 26 kB 00:00 2024-09-05T01:03:12.541 INFO:teuthology.orchestra.run.smithi129.stdout:(96/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 474 kB/s | 47 kB 00:00 2024-09-05T01:03:12.555 INFO:teuthology.orchestra.run.smithi183.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-05T01:03:12.569 INFO:teuthology.orchestra.run.smithi194.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-05T01:03:12.589 INFO:teuthology.orchestra.run.smithi183.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-05T01:03:12.614 INFO:teuthology.orchestra.run.smithi183.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-05T01:03:12.637 INFO:teuthology.orchestra.run.smithi194.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 303 kB/s | 36 kB 00:00 2024-09-05T01:03:12.650 INFO:teuthology.orchestra.run.smithi129.stdout:(97/154): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-05T01:03:12.656 INFO:teuthology.orchestra.run.smithi183.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-05T01:03:12.661 INFO:teuthology.orchestra.run.smithi194.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 517 kB/s | 47 kB 00:00 2024-09-05T01:03:12.681 INFO:teuthology.orchestra.run.smithi183.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 3.2 MB/s | 222 kB 00:00 2024-09-05T01:03:12.728 INFO:teuthology.orchestra.run.smithi194.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 3.3 MB/s | 304 kB 00:00 2024-09-05T01:03:12.748 INFO:teuthology.orchestra.run.smithi183.stdout:(85/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 5.1 MB 00:00 2024-09-05T01:03:12.750 INFO:teuthology.orchestra.run.smithi129.stdout:(98/154): uuid-1.6.2-55.el9.x86_64.rpm 579 kB/s | 58 kB 00:00 2024-09-05T01:03:12.753 INFO:teuthology.orchestra.run.smithi194.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 630 kB/s | 58 kB 00:00 2024-09-05T01:03:12.773 INFO:teuthology.orchestra.run.smithi183.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-05T01:03:12.799 INFO:teuthology.orchestra.run.smithi183.stdout:(87/154): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 77 kB 00:00 2024-09-05T01:03:12.832 INFO:teuthology.orchestra.run.smithi183.stdout:(88/154): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-09-05T01:03:12.859 INFO:teuthology.orchestra.run.smithi129.stdout:(99/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 884 kB/s | 96 kB 00:00 2024-09-05T01:03:12.866 INFO:teuthology.orchestra.run.smithi183.stdout:(89/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-05T01:03:12.891 INFO:teuthology.orchestra.run.smithi183.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-05T01:03:12.917 INFO:teuthology.orchestra.run.smithi183.stdout:(91/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-05T01:03:12.920 INFO:teuthology.orchestra.run.smithi194.stdout:(100/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-05T01:03:12.942 INFO:teuthology.orchestra.run.smithi183.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-05T01:03:12.945 INFO:teuthology.orchestra.run.smithi194.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-09-05T01:03:12.959 INFO:teuthology.orchestra.run.smithi129.stdout:(100/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-09-05T01:03:12.970 INFO:teuthology.orchestra.run.smithi194.stdout:(102/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 396 kB/s | 96 kB 00:00 2024-09-05T01:03:12.984 INFO:teuthology.orchestra.run.smithi183.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 622 kB/s | 42 kB 00:00 2024-09-05T01:03:12.996 INFO:teuthology.orchestra.run.smithi194.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-09-05T01:03:13.009 INFO:teuthology.orchestra.run.smithi183.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 660 kB/s | 44 kB 00:00 2024-09-05T01:03:13.046 INFO:teuthology.orchestra.run.smithi194.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T01:03:13.059 INFO:teuthology.orchestra.run.smithi129.stdout:(101/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 448 kB/s | 45 kB 00:00 2024-09-05T01:03:13.060 INFO:teuthology.orchestra.run.smithi183.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-05T01:03:13.071 INFO:teuthology.orchestra.run.smithi194.stdout:(105/154): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-05T01:03:13.085 INFO:teuthology.orchestra.run.smithi183.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 477 kB/s | 36 kB 00:00 2024-09-05T01:03:13.105 INFO:teuthology.orchestra.run.smithi194.stdout:(106/154): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-09-05T01:03:13.127 INFO:teuthology.orchestra.run.smithi183.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-09-05T01:03:13.130 INFO:teuthology.orchestra.run.smithi194.stdout:(107/154): grpc-data-1.46.7-10.el9.noarch.rpm 331 kB/s | 19 kB 00:00 2024-09-05T01:03:13.135 INFO:teuthology.orchestra.run.smithi129.stdout:(102/154): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-05T01:03:13.152 INFO:teuthology.orchestra.run.smithi183.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-05T01:03:13.155 INFO:teuthology.orchestra.run.smithi194.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T01:03:13.181 INFO:teuthology.orchestra.run.smithi194.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-05T01:03:13.206 INFO:teuthology.orchestra.run.smithi194.stdout:(110/154): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-05T01:03:13.240 INFO:teuthology.orchestra.run.smithi194.stdout:(111/154): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-05T01:03:13.273 INFO:teuthology.orchestra.run.smithi194.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-05T01:03:13.277 INFO:teuthology.orchestra.run.smithi129.stdout:(103/154): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-09-05T01:03:13.323 INFO:teuthology.orchestra.run.smithi194.stdout:(113/154): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-05T01:03:13.327 INFO:teuthology.orchestra.run.smithi183.stdout:(99/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-05T01:03:13.335 INFO:teuthology.orchestra.run.smithi129.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-05T01:03:13.352 INFO:teuthology.orchestra.run.smithi183.stdout:(100/154): uuid-1.6.2-55.el9.x86_64.rpm 256 kB/s | 58 kB 00:00 2024-09-05T01:03:13.365 INFO:teuthology.orchestra.run.smithi194.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-05T01:03:13.369 INFO:teuthology.orchestra.run.smithi129.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-05T01:03:13.378 INFO:teuthology.orchestra.run.smithi183.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 425 kB/s | 96 kB 00:00 2024-09-05T01:03:13.391 INFO:teuthology.orchestra.run.smithi194.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T01:03:13.394 INFO:teuthology.orchestra.run.smithi129.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-05T01:03:13.406 INFO:teuthology.orchestra.run.smithi183.stdout:(102/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 816 kB/s | 64 kB 00:00 2024-09-05T01:03:13.424 INFO:teuthology.orchestra.run.smithi194.stdout:(116/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-05T01:03:13.436 INFO:teuthology.orchestra.run.smithi183.stdout:(103/154): lua-devel-5.4.4-4.el9.x86_64.rpm 382 kB/s | 22 kB 00:00 2024-09-05T01:03:13.450 INFO:teuthology.orchestra.run.smithi194.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 242 kB/s | 14 kB 00:00 2024-09-05T01:03:13.462 INFO:teuthology.orchestra.run.smithi183.stdout:(104/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 411 kB/s | 45 kB 00:00 2024-09-05T01:03:13.483 INFO:teuthology.orchestra.run.smithi194.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-05T01:03:13.495 INFO:teuthology.orchestra.run.smithi129.stdout:(107/154): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-05T01:03:13.520 INFO:teuthology.orchestra.run.smithi129.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T01:03:13.521 INFO:teuthology.orchestra.run.smithi183.stdout:(105/154): protobuf-compiler-3.14.0-13.el9.x86_ 7.4 MB/s | 863 kB 00:00 2024-09-05T01:03:13.545 INFO:teuthology.orchestra.run.smithi129.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-05T01:03:13.550 INFO:teuthology.orchestra.run.smithi194.stdout:(119/154): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-05T01:03:13.562 INFO:teuthology.orchestra.run.smithi183.stdout:(106/154): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-09-05T01:03:13.575 INFO:teuthology.orchestra.run.smithi194.stdout:(120/154): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-05T01:03:13.587 INFO:teuthology.orchestra.run.smithi129.stdout:(110/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-05T01:03:13.588 INFO:teuthology.orchestra.run.smithi183.stdout:(107/154): abseil-cpp-20211102.0-3.el9.x86_64.r 3.5 MB/s | 548 kB 00:00 2024-09-05T01:03:13.601 INFO:teuthology.orchestra.run.smithi194.stdout:(121/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-05T01:03:13.612 INFO:teuthology.orchestra.run.smithi129.stdout:(111/154): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-05T01:03:13.613 INFO:teuthology.orchestra.run.smithi183.stdout:(108/154): grpc-data-1.46.7-10.el9.noarch.rpm 214 kB/s | 19 kB 00:00 2024-09-05T01:03:13.639 INFO:teuthology.orchestra.run.smithi183.stdout:(109/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 517 kB/s | 26 kB 00:00 2024-09-05T01:03:13.640 INFO:teuthology.orchestra.run.smithi194.stdout:(122/154): python3-google-auth-2.34.0-1.el9.noa 2.4 MB/s | 220 kB 00:00 2024-09-05T01:03:13.646 INFO:teuthology.orchestra.run.smithi129.stdout:(112/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-09-05T01:03:13.660 INFO:teuthology.orchestra.run.smithi194.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 553 kB/s | 11 kB 00:00 2024-09-05T01:03:13.664 INFO:teuthology.orchestra.run.smithi183.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-05T01:03:13.685 INFO:teuthology.orchestra.run.smithi194.stdout:(124/154): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-05T01:03:13.688 INFO:teuthology.orchestra.run.smithi129.stdout:(113/154): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-05T01:03:13.689 INFO:teuthology.orchestra.run.smithi183.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-05T01:03:13.710 INFO:teuthology.orchestra.run.smithi194.stdout:(125/154): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-05T01:03:13.713 INFO:teuthology.orchestra.run.smithi129.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T01:03:13.714 INFO:teuthology.orchestra.run.smithi183.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-05T01:03:13.736 INFO:teuthology.orchestra.run.smithi194.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-05T01:03:13.739 INFO:teuthology.orchestra.run.smithi129.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T01:03:13.756 INFO:teuthology.orchestra.run.smithi183.stdout:(113/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-05T01:03:13.761 INFO:teuthology.orchestra.run.smithi194.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-05T01:03:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-05T01:03:13.786 INFO:teuthology.orchestra.run.smithi194.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-05T01:03:13.789 INFO:teuthology.orchestra.run.smithi129.stdout:(117/154): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-05T01:03:13.807 INFO:teuthology.orchestra.run.smithi183.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-09-05T01:03:13.811 INFO:teuthology.orchestra.run.smithi194.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T01:03:13.823 INFO:teuthology.orchestra.run.smithi129.stdout:(118/154): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-05T01:03:13.823 INFO:teuthology.orchestra.run.smithi183.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-05T01:03:13.845 INFO:teuthology.orchestra.run.smithi194.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-05T01:03:13.857 INFO:teuthology.orchestra.run.smithi129.stdout:(119/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-05T01:03:13.857 INFO:teuthology.orchestra.run.smithi183.stdout:(116/154): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-09-05T01:03:13.870 INFO:teuthology.orchestra.run.smithi194.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T01:03:13.874 INFO:teuthology.orchestra.run.smithi183.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-05T01:03:13.882 INFO:teuthology.orchestra.run.smithi129.stdout:(120/154): python3-google-auth-2.34.0-1.el9.noa 8.7 MB/s | 220 kB 00:00 2024-09-05T01:03:13.899 INFO:teuthology.orchestra.run.smithi183.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-05T01:03:13.904 INFO:teuthology.orchestra.run.smithi194.stdout:(132/154): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-05T01:03:13.929 INFO:teuthology.orchestra.run.smithi194.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-05T01:03:13.933 INFO:teuthology.orchestra.run.smithi183.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T01:03:13.949 INFO:teuthology.orchestra.run.smithi129.stdout:(121/154): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2024-09-05T01:03:13.963 INFO:teuthology.orchestra.run.smithi194.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-05T01:03:13.966 INFO:teuthology.orchestra.run.smithi183.stdout:(120/154): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-05T01:03:13.974 INFO:teuthology.orchestra.run.smithi129.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-05T01:03:13.988 INFO:teuthology.orchestra.run.smithi194.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-05T01:03:13.992 INFO:teuthology.orchestra.run.smithi183.stdout:(121/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-05T01:03:14.000 INFO:teuthology.orchestra.run.smithi129.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-09-05T01:03:14.013 INFO:teuthology.orchestra.run.smithi194.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-05T01:03:14.025 INFO:teuthology.orchestra.run.smithi129.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-05T01:03:14.025 INFO:teuthology.orchestra.run.smithi183.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-05T01:03:14.039 INFO:teuthology.orchestra.run.smithi194.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-05T01:03:14.042 INFO:teuthology.orchestra.run.smithi183.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 642 kB/s | 11 kB 00:00 2024-09-05T01:03:14.050 INFO:teuthology.orchestra.run.smithi129.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-05T01:03:14.059 INFO:teuthology.orchestra.run.smithi183.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 1.1 MB/s | 18 kB 00:00 2024-09-05T01:03:14.075 INFO:teuthology.orchestra.run.smithi129.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-05T01:03:14.076 INFO:teuthology.orchestra.run.smithi183.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-05T01:03:14.089 INFO:teuthology.orchestra.run.smithi194.stdout:(138/154): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-05T01:03:14.093 INFO:teuthology.orchestra.run.smithi183.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 1.2 MB/s | 19 kB 00:00 2024-09-05T01:03:14.109 INFO:teuthology.orchestra.run.smithi129.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-09-05T01:03:14.110 INFO:teuthology.orchestra.run.smithi183.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-05T01:03:14.114 INFO:teuthology.orchestra.run.smithi194.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 789 kB/s | 59 kB 00:00 2024-09-05T01:03:14.139 INFO:teuthology.orchestra.run.smithi194.stdout:(140/154): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-05T01:03:14.160 INFO:teuthology.orchestra.run.smithi129.stdout:(128/154): python3-kubernetes-26.1.0-2.el9.noar 20 MB/s | 1.0 MB 00:00 2024-09-05T01:03:14.164 INFO:teuthology.orchestra.run.smithi194.stdout:(141/154): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-05T01:03:14.177 INFO:teuthology.orchestra.run.smithi129.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-05T01:03:14.177 INFO:teuthology.orchestra.run.smithi183.stdout:(128/154): libarrow-9.0.0-11.el9.x86_64.rpm 7.2 MB/s | 4.4 MB 00:00 2024-09-05T01:03:14.190 INFO:teuthology.orchestra.run.smithi194.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 989 kB/s | 74 kB 00:00 2024-09-05T01:03:14.202 INFO:teuthology.orchestra.run.smithi129.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T01:03:14.215 INFO:teuthology.orchestra.run.smithi194.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-05T01:03:14.219 INFO:teuthology.orchestra.run.smithi183.stdout:(129/154): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-05T01:03:14.219 INFO:teuthology.orchestra.run.smithi129.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-05T01:03:14.240 INFO:teuthology.orchestra.run.smithi194.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-05T01:03:14.244 INFO:teuthology.orchestra.run.smithi183.stdout:(130/154): python3-logutils-0.3.5-21.el9.noarch 693 kB/s | 46 kB 00:00 2024-09-05T01:03:14.252 INFO:teuthology.orchestra.run.smithi129.stdout:(132/154): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-05T01:03:14.266 INFO:teuthology.orchestra.run.smithi194.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-05T01:03:14.270 INFO:teuthology.orchestra.run.smithi183.stdout:(131/154): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-05T01:03:14.280 INFO:teuthology.orchestra.run.smithi129.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 655 kB/s | 16 kB 00:00 2024-09-05T01:03:14.291 INFO:teuthology.orchestra.run.smithi194.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T01:03:14.295 INFO:teuthology.orchestra.run.smithi183.stdout:(132/154): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-05T01:03:14.303 INFO:teuthology.orchestra.run.smithi129.stdout:(134/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-05T01:03:14.316 INFO:teuthology.orchestra.run.smithi194.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-05T01:03:14.320 INFO:teuthology.orchestra.run.smithi183.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-05T01:03:14.329 INFO:teuthology.orchestra.run.smithi129.stdout:(135/154): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-05T01:03:14.345 INFO:teuthology.orchestra.run.smithi183.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-05T01:03:14.354 INFO:teuthology.orchestra.run.smithi129.stdout:(136/154): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-05T01:03:14.367 INFO:teuthology.orchestra.run.smithi194.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-05T01:03:14.371 INFO:teuthology.orchestra.run.smithi183.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-05T01:03:14.379 INFO:teuthology.orchestra.run.smithi129.stdout:(137/154): python3-routes-2.5.1-5.el9.noarch.rp 7.4 MB/s | 188 kB 00:00 2024-09-05T01:03:14.396 INFO:teuthology.orchestra.run.smithi183.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-05T01:03:14.404 INFO:teuthology.orchestra.run.smithi129.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-05T01:03:14.421 INFO:teuthology.orchestra.run.smithi183.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 609 kB/s | 31 kB 00:00 2024-09-05T01:03:14.430 INFO:teuthology.orchestra.run.smithi129.stdout:(139/154): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-05T01:03:14.447 INFO:teuthology.orchestra.run.smithi183.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-05T01:03:14.463 INFO:teuthology.orchestra.run.smithi129.stdout:(140/154): python3-typing-extensions-4.12.2-2.e 2.2 MB/s | 74 kB 00:00 2024-09-05T01:03:14.480 INFO:teuthology.orchestra.run.smithi183.stdout:(139/154): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2024-09-05T01:03:14.489 INFO:teuthology.orchestra.run.smithi129.stdout:(141/154): python3-webob-1.8.8-2.el9.noarch.rpm 9.1 MB/s | 230 kB 00:00 2024-09-05T01:03:14.505 INFO:teuthology.orchestra.run.smithi183.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-09-05T01:03:14.514 INFO:teuthology.orchestra.run.smithi129.stdout:(142/154): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-05T01:03:14.531 INFO:teuthology.orchestra.run.smithi183.stdout:(141/154): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-05T01:03:14.548 INFO:teuthology.orchestra.run.smithi129.stdout:(143/154): python3-werkzeug-2.0.3-3.el9.1.noarc 13 MB/s | 427 kB 00:00 2024-09-05T01:03:14.556 INFO:teuthology.orchestra.run.smithi183.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-05T01:03:14.558 INFO:teuthology.orchestra.run.smithi194.stdout:(149/154): NetworkManager-1.48.10-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-09-05T01:03:14.573 INFO:teuthology.orchestra.run.smithi129.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 798 kB/s | 20 kB 00:00 2024-09-05T01:03:14.590 INFO:teuthology.orchestra.run.smithi183.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-05T01:03:14.598 INFO:teuthology.orchestra.run.smithi129.stdout:(145/154): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-05T01:03:14.615 INFO:teuthology.orchestra.run.smithi183.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-05T01:03:14.625 INFO:teuthology.orchestra.run.smithi194.stdout:(150/154): librados2-19.1.1-259.g91c8a7e5.el9.x 10 MB/s | 3.4 MB 00:00 2024-09-05T01:03:14.640 INFO:teuthology.orchestra.run.smithi183.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-05T01:03:14.657 INFO:teuthology.orchestra.run.smithi129.stdout:(146/154): thrift-0.15.0-2.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-09-05T01:03:14.665 INFO:teuthology.orchestra.run.smithi183.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T01:03:14.676 INFO:teuthology.orchestra.run.smithi194.stdout:(151/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-05T01:03:14.699 INFO:teuthology.orchestra.run.smithi183.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-05T01:03:14.717 INFO:teuthology.orchestra.run.smithi194.stdout:(152/154): NetworkManager-libnm-1.48.10-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-09-05T01:03:14.743 INFO:teuthology.orchestra.run.smithi194.stdout:(153/154): NetworkManager-team-1.48.10-1.el9.x8 347 kB/s | 40 kB 00:00 2024-09-05T01:03:14.749 INFO:teuthology.orchestra.run.smithi183.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T01:03:14.843 INFO:teuthology.orchestra.run.smithi194.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 1.5 MB/s | 248 kB 00:00 2024-09-05T01:03:14.849 INFO:teuthology.orchestra.run.smithi194.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:03:14.849 INFO:teuthology.orchestra.run.smithi194.stdout:Total 21 MB/s | 222 MB 00:10 2024-09-05T01:03:14.916 INFO:teuthology.orchestra.run.smithi129.stdout:(147/154): librados2-19.1.1-259.g91c8a7e5.el9.x 13 MB/s | 3.4 MB 00:00 2024-09-05T01:03:14.992 INFO:teuthology.orchestra.run.smithi183.stdout:(149/154): librados2-19.1.1-259.g91c8a7e5.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-05T01:03:15.049 INFO:teuthology.orchestra.run.smithi129.stdout:(148/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 24 MB/s | 3.2 MB 00:00 2024-09-05T01:03:15.058 INFO:teuthology.orchestra.run.smithi183.stdout:(150/154): librbd1-19.1.1-259.g91c8a7e5.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-05T01:03:15.117 INFO:teuthology.orchestra.run.smithi183.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 698 kB/s | 40 kB 00:00 2024-09-05T01:03:15.201 INFO:teuthology.orchestra.run.smithi183.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-09-05T01:03:15.292 INFO:teuthology.orchestra.run.smithi183.stdout:(153/154): NetworkManager-1.48.10-1.el9.x86_64. 4.3 MB/s | 2.3 MB 00:00 2024-09-05T01:03:15.366 INFO:teuthology.orchestra.run.smithi129.stdout:(149/154): NetworkManager-1.48.10-1.el9.x86_64. 7.3 MB/s | 2.3 MB 00:00 2024-09-05T01:03:15.392 INFO:teuthology.orchestra.run.smithi183.stdout:(154/154): NetworkManager-libnm-1.48.10-1.el9.x 4.5 MB/s | 1.8 MB 00:00 2024-09-05T01:03:15.399 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:03:15.399 INFO:teuthology.orchestra.run.smithi183.stdout:Total 21 MB/s | 222 MB 00:10 2024-09-05T01:03:15.442 INFO:teuthology.orchestra.run.smithi129.stdout:(150/154): NetworkManager-libnm-1.48.10-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-05T01:03:15.475 INFO:teuthology.orchestra.run.smithi129.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 1.2 MB/s | 40 kB 00:00 2024-09-05T01:03:15.517 INFO:teuthology.orchestra.run.smithi129.stdout:(152/154): NetworkManager-tui-1.48.10-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-09-05T01:03:16.038 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction check 2024-09-05T01:03:16.172 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction check succeeded. 2024-09-05T01:03:16.173 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction test 2024-09-05T01:03:16.463 INFO:teuthology.orchestra.run.smithi129.stdout:(153/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.6 MB/s | 16 MB 00:04 2024-09-05T01:03:16.578 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-09-05T01:03:16.713 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-09-05T01:03:16.713 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-09-05T01:03:17.935 INFO:teuthology.orchestra.run.smithi194.stdout:Transaction test succeeded. 2024-09-05T01:03:17.935 INFO:teuthology.orchestra.run.smithi194.stdout:Running transaction 2024-09-05T01:03:18.310 INFO:teuthology.orchestra.run.smithi129.stdout:(154/154): python3-numpy-1.20.1-5.el9.x86_64.rp 750 kB/s | 5.1 MB 00:06 2024-09-05T01:03:18.313 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:03:18.313 INFO:teuthology.orchestra.run.smithi129.stdout:Total 14 MB/s | 222 MB 00:16 2024-09-05T01:03:18.459 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-09-05T01:03:18.459 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-09-05T01:03:19.475 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-05T01:03:19.610 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-05T01:03:19.610 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-05T01:03:21.356 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-05T01:03:21.357 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-05T01:03:24.086 INFO:teuthology.orchestra.run.smithi194.stdout: Preparing : 1/1 2024-09-05T01:03:24.153 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:03:24.187 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:03:24.230 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:03:24.569 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:03:24.610 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-09-05T01:03:24.679 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:24.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:03:24.785 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:24.804 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:03:24.814 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:24.908 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:24.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:03:25.023 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:03:25.051 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:03:25.077 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:03:25.233 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:03:25.282 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:03:25.309 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:03:25.319 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:25.422 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:25.429 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:25.445 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:25.507 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:25.529 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:25.552 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:25.553 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:25.597 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:25.689 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:25.730 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:03:25.747 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:25.760 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:03:25.786 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:03:25.910 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:25.938 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:03:25.983 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:03:26.011 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:26.026 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:26.086 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:26.123 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:26.146 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:26.189 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:26.208 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:26.239 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:26.262 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:26.271 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:26.457 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:26.507 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:26.637 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:03:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:26.733 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:03:26.767 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:03:26.769 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:26.802 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:03:26.829 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:26.876 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:03:26.928 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:03:26.934 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:26.968 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:03:27.012 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:03:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:27.053 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:03:27.089 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:03:27.158 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:03:27.218 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:03:27.235 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:27.250 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:03:27.293 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:03:27.352 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:03:27.370 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:03:27.404 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:03:27.447 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:03:27.458 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:03:27.476 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:03:27.494 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:03:27.529 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:03:27.548 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:03:27.599 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:03:27.603 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:03:27.617 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-05T01:03:27.655 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:03:27.675 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:03:27.696 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:03:27.738 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T01:03:27.740 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:03:27.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:03:27.812 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:03:27.817 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:03:27.869 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:03:27.889 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:03:27.939 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T01:03:27.948 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:03:27.978 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:03:27.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T01:03:28.022 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:03:28.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:03:28.141 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:03:28.185 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:03:28.213 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:03:28.286 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:03:28.313 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T01:03:28.332 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:03:28.405 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:03:28.408 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:03:28.430 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:28.484 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:03:28.521 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:03:28.536 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 5/160 2024-09-05T01:03:28.555 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:03:28.564 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:03:28.565 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:28.602 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:03:28.627 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:03:28.634 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:03:28.690 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:03:28.764 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:03:28.764 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:03:28.794 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:03:28.806 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:03:28.830 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:03:28.836 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T01:03:28.862 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T01:03:28.877 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:03:28.913 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:03:28.964 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:03:29.014 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T01:03:29.015 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:03:29.067 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T01:03:29.112 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:29.169 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:03:29.170 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:03:29.200 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:03:29.208 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:03:29.230 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:29.247 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:03:29.249 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:03:29.284 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 13/160 2024-09-05T01:03:29.291 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:03:29.314 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:29.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:03:29.340 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T01:03:29.353 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:03:29.373 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:03:29.406 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:03:29.458 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:03:29.462 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:03:29.535 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:03:29.565 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:03:29.573 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:03:29.601 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:03:29.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:03:29.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:03:29.745 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:03:29.767 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T01:03:29.795 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:03:29.825 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:03:29.879 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:29.932 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:03:29.935 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:03:29.939 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T01:03:29.980 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:03:30.021 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:03:30.021 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:03:30.054 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:30.127 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:03:30.149 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:30.163 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:03:30.205 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:03:30.230 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:03:30.254 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:03:30.330 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:03:30.358 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:03:30.367 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T01:03:30.396 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:03:30.424 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:03:30.458 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:03:30.479 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T01:03:30.503 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:03:30.540 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:03:30.580 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:03:30.584 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T01:03:30.618 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:03:30.621 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T01:03:30.630 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:03:30.656 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T01:03:30.660 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:03:30.664 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:03:30.692 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:03:30.694 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:03:30.729 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T01:03:30.735 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:03:30.736 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:03:30.770 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:03:30.773 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T01:03:30.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:03:30.822 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T01:03:30.825 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:03:30.858 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:03:30.865 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:03:30.874 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T01:03:30.903 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:03:30.907 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T01:03:30.909 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:03:30.935 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T01:03:30.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:03:30.940 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:03:30.971 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:03:31.003 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T01:03:31.021 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:03:31.045 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:03:31.055 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:03:31.057 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T01:03:31.088 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T01:03:31.108 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:03:31.123 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T01:03:31.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:03:31.200 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T01:03:31.203 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:03:31.234 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:03:31.243 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 33/160 2024-09-05T01:03:31.286 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:03:31.315 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 35/160 2024-09-05T01:03:31.339 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:03:31.386 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T01:03:31.402 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:03:31.430 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T01:03:31.506 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T01:03:31.638 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T01:03:31.691 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T01:03:31.862 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:03:31.904 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:03:31.938 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:03:32.042 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:03:32.093 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:03:32.147 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:03:32.148 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:03:32.198 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:03:32.229 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T01:03:32.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:03:32.236 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:03:32.316 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T01:03:32.327 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:03:32.334 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:03:32.344 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T01:03:32.387 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T01:03:32.396 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:03:32.426 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 45/160 2024-09-05T01:03:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:03:32.459 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T01:03:32.507 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:03:32.515 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T01:03:32.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:03:32.538 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:03:32.577 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:03:32.587 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T01:03:32.615 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:03:32.627 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T01:03:32.663 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T01:03:32.710 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T01:03:32.746 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T01:03:32.759 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:03:32.789 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T01:03:32.791 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:03:32.832 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T01:03:32.869 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:03:32.968 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T01:03:33.017 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T01:03:33.059 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T01:03:33.170 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T01:03:33.284 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:03:33.360 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T01:03:33.400 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T01:03:33.406 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:03:33.433 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T01:03:33.458 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T01:03:33.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T01:03:33.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:03:33.649 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:03:33.682 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T01:03:33.908 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T01:03:34.037 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T01:03:34.167 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T01:03:34.211 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T01:03:34.249 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T01:03:34.288 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T01:03:34.327 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T01:03:34.361 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T01:03:34.394 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T01:03:34.436 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T01:03:34.471 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T01:03:34.558 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T01:03:34.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T01:03:34.640 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T01:03:34.666 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:03:34.743 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:03:34.751 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T01:03:34.771 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:03:34.803 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:03:34.808 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T01:03:34.906 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:03:34.995 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:03:35.032 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:03:35.073 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:03:35.182 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:03:35.258 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:35.357 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:35.387 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:03:35.390 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:03:35.432 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:03:35.509 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:35.541 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T01:03:35.589 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T01:03:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:35.622 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T01:03:35.649 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:03:35.693 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:03:35.722 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T01:03:35.778 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T01:03:35.823 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T01:03:35.911 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T01:03:35.921 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:03:35.967 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:35.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T01:03:36.041 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:36.071 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:03:36.168 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T01:03:36.174 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:03:36.198 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T01:03:36.205 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:36.276 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T01:03:36.277 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:36.307 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:03:36.938 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T01:03:37.056 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T01:03:38.297 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T01:03:38.388 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T01:03:38.396 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:38.415 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T01:03:38.441 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:38.449 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T01:03:38.759 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T01:03:38.805 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:38.860 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:38.896 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 99/160 2024-09-05T01:03:38.904 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:38.927 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 100/160 2024-09-05T01:03:38.963 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T01:03:38.989 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:39.036 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:39.128 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:39.138 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:03:39.138 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:03:39.138 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:39.162 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:39.450 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:39.455 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T01:03:39.490 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:39.492 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:39.571 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 103/160 2024-09-05T01:03:39.578 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:39.578 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:03:39.578 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:03:39.578 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:39.602 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 104/160 2024-09-05T01:03:39.617 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:41.946 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:41.975 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:42.522 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 105/160 2024-09-05T01:03:42.562 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:42.644 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 106/160 2024-09-05T01:03:42.644 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T01:03:42.644 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T01:03:42.644 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:42.672 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:50.564 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:50.564 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /sys 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /proc 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /mnt 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /var/tmp 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /home 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /root 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout:skipping the directory /tmp 2024-09-05T01:03:50.565 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:50.739 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:50.759 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:50.759 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:03:50.759 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-09-05T01:03:51.282 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-09-05T01:03:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:51.470 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:51.486 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:51.486 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:51.486 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:03:51.486 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:52.217 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:52.244 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:52.475 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:52.499 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:52.500 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:52.500 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:03:52.500 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:52.500 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:52.500 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:52.934 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:52.958 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:52.965 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:52.992 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:53.064 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:03:53.121 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:03:53.177 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:03:53.215 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:53.239 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:53.364 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:03:53.410 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:03:53.487 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:03:53.576 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:03:53.675 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:53.698 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:53.699 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:53.699 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:03:53.699 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:53.699 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:53.699 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:53.717 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:03:53.760 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:03:53.792 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:03:53.804 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:03:53.828 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:03:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:03:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:03:54.099 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:03:54.150 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:03:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:03:54.257 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:54.292 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:54.324 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:03:54.332 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:03:54.394 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 107/160 2024-09-05T01:03:54.394 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:54.395 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:03:54.424 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:54.463 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:03:54.513 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:03:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:03:54.570 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:54.586 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:03:54.587 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 108/160 2024-09-05T01:03:54.588 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:54.588 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T01:03:54.588 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:55.039 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:55.067 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:55.098 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:03:55.160 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:03:55.190 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:55.466 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:55.500 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:03:56.065 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 109/160 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T01:03:56.092 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:56.229 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:56.267 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:03:56.324 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 110/160 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T01:03:56.348 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:56.463 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:03:56.492 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:03:56.596 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:03:56.755 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:03:56.783 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:56.786 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:03:56.806 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 111/160 2024-09-05T01:03:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:56.806 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T01:03:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:56.807 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T01:03:56.807 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:03:56.810 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:03:56.810 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:56.811 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:03:56.811 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:03:56.811 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:03:56.811 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:56.862 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:03:56.874 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:03:56.904 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T01:03:56.953 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T01:03:56.981 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:03:57.009 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T01:03:57.035 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:03:57.200 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T01:03:57.225 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:03:57.251 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:03:57.251 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T01:03:57.328 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T01:03:57.355 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:03:57.416 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T01:03:57.522 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:03:57.553 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:03:57.574 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:03:57.577 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:57.609 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T01:03:57.629 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:03:57.641 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:03:57.641 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T01:03:57.670 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T01:03:57.737 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:03:57.793 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:03:57.959 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:03:57.989 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:03:58.014 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:03:58.015 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.015 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:03:58.015 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:03:58.015 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:03:58.015 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:58.057 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:03:58.080 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:03:58.080 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.080 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:03:58.080 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:58.090 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:58.118 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 123/160 2024-09-05T01:03:58.124 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:03:58.147 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:03:58.148 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.148 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:03:58.148 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:03:58.148 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:03:58.148 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:58.149 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 124/160 2024-09-05T01:03:58.213 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 125/160 2024-09-05T01:03:58.242 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:58.449 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:03:58.473 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:03:58.473 INFO:teuthology.orchestra.run.smithi194.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.473 INFO:teuthology.orchestra.run.smithi194.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:03:58.473 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:03:58.473 INFO:teuthology.orchestra.run.smithi194.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:03:58.474 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:03:58.717 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:03:58.748 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:03:58.772 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:03:58.772 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.772 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:03:58.773 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:03:58.773 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:03:58.773 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:58.824 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:03:58.847 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:03:58.847 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.847 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:03:58.847 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:58.900 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:03:58.922 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:03:58.922 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:58.922 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:03:58.923 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:03:58.923 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:03:58.923 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:59.225 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:03:59.248 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:03:59.249 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:03:59.249 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:03:59.249 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:03:59.249 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:03:59.249 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:03:59.286 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 126/160 2024-09-05T01:03:59.327 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:04:00.296 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 127/160 2024-09-05T01:04:00.327 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:04:00.428 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 128/160 2024-09-05T01:04:00.590 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 129/160 2024-09-05T01:04:00.621 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:04:00.645 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 130/160 2024-09-05T01:04:00.645 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:04:00.646 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T01:04:00.646 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:04:00.646 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T01:04:00.646 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:00.697 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:04:00.708 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 131/160 2024-09-05T01:04:00.807 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 132/160 2024-09-05T01:04:00.862 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 133/160 2024-09-05T01:04:01.788 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T01:04:01.825 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 135/160 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T01:04:01.848 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:01.900 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:04:01.923 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 136/160 2024-09-05T01:04:01.923 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:04:01.924 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T01:04:01.924 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:01.976 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:04:01.999 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 137/160 2024-09-05T01:04:01.999 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:04:01.999 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T01:04:02.000 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:04:02.000 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T01:04:02.000 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:02.302 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:04:02.325 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 138/160 2024-09-05T01:04:02.325 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T01:04:02.325 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T01:04:02.325 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:04:02.325 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T01:04:02.326 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:02.994 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:04:03.035 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:04:03.084 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:04:03.117 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:04:03.144 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:04:03.328 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:04:03.374 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:04:03.404 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:04:03.434 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:04:03.480 INFO:teuthology.orchestra.run.smithi194.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:04:03.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:04:03.829 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:04:03.868 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:04:03.901 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:04:03.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:04:04.101 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.117 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:04:04.135 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.156 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.158 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:04:04.189 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:04:04.218 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:04:04.265 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:04:04.921 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.963 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.983 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:04.990 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:04:05.073 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:04:05.114 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:04:05.195 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:04:05.226 INFO:teuthology.orchestra.run.smithi194.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:04:05.252 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:04:05.252 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:05.344 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:05.369 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:04:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:05.375 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:05.546 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:05.638 INFO:teuthology.orchestra.run.smithi194.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:05.830 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:04:05.926 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:04:05.967 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:04:06.047 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:04:06.079 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:04:06.105 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:04:06.105 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:06.189 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:06.214 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:04:06.220 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:06.220 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:06.400 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:06.400 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:06.484 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:06.484 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:06.864 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 139/160 2024-09-05T01:04:06.905 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 140/160 2024-09-05T01:04:06.946 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 141/160 2024-09-05T01:04:06.987 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T01:04:07.023 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T01:04:07.200 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 144/160 2024-09-05T01:04:07.253 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T01:04:07.284 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 146/160 2024-09-05T01:04:07.313 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T01:04:07.360 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T01:04:07.984 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:08.025 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:08.045 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T01:04:08.872 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 150/160 2024-09-05T01:04:08.986 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T01:04:09.030 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T01:04:09.111 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T01:04:09.144 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T01:04:09.168 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T01:04:09.169 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:09.244 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T01:04:09.269 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T01:04:09.274 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:09.275 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:09.454 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:09.455 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:09.530 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T01:04:09.530 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:11.472 INFO:teuthology.orchestra.run.smithi194.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:11.472 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:04:11.473 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:04:11.474 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:04:11.475 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:04:11.475 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:04:11.475 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:04:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:04:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:04:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:04:11.486 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:04:11.487 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:04:11.488 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:04:11.489 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:04:11.490 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:04:11.491 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:04:11.492 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:04:11.493 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:04:11.494 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:04:11.495 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:04:11.496 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:11.497 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:04:12.286 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:04:12.287 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:04:12.288 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:04:12.300 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:04:12.300 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:04:12.300 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:04:12.301 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:04:12.302 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:04:12.303 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:04:12.304 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:04:12.305 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:04:12.306 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:04:12.307 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:04:12.308 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:04:12.309 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:12.310 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout:Upgraded: 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:13.978 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout:Installed: 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.979 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.980 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:04:13.981 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.982 INFO:teuthology.orchestra.run.smithi194.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:04:13.983 INFO:teuthology.orchestra.run.smithi194.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:04:13.984 INFO:teuthology.orchestra.run.smithi194.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:04:13.985 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:04:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:04:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:04:13.986 INFO:teuthology.orchestra.run.smithi194.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:04:13.987 INFO:teuthology.orchestra.run.smithi194.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:04:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:04:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:04:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:04:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:04:13.988 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:04:13.989 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:04:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:04:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:04:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:04:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:04:13.990 INFO:teuthology.orchestra.run.smithi194.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:04:13.991 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:04:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:04:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:04:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:04:13.992 INFO:teuthology.orchestra.run.smithi194.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:04:13.993 INFO:teuthology.orchestra.run.smithi194.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:04:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:04:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:04:13.994 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:04:13.995 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.996 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:04:13.997 INFO:teuthology.orchestra.run.smithi194.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:04:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:04:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:04:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:04:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:04:13.998 INFO:teuthology.orchestra.run.smithi194.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:04:13.999 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:04:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:04:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:04:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.000 INFO:teuthology.orchestra.run.smithi194.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:04:14.001 INFO:teuthology.orchestra.run.smithi194.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:04:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:04:14.002 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:04:14.003 INFO:teuthology.orchestra.run.smithi194.stdout:Complete! 2024-09-05T01:04:14.359 DEBUG:teuthology.parallel:result is None 2024-09-05T01:04:14.923 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-09-05T01:04:14.924 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.925 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.926 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:04:14.927 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.928 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:04:14.929 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:04:14.930 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:04:14.931 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:04:14.932 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:04:14.933 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.934 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:04:14.935 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:04:14.936 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:04:14.937 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:04:14.937 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:04:14.937 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-09-05T01:04:15.327 DEBUG:teuthology.parallel:result is None 2024-09-05T01:04:15.521 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T01:04:15.521 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 1/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 3/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 4/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-259.g91c8a7 5/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 6/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 7/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 8/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 9/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 10/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 11/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 12/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 13/160 2024-09-05T01:04:15.522 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 14/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_6 15/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 16/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 17/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 18/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_ 19/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 20/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9 21/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x 22/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 23/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 24/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 25/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 26/160 2024-09-05T01:04:15.523 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 27/160 2024-09-05T01:04:15.524 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 28/160 2024-09-05T01:04:15.524 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 29/160 2024-09-05T01:04:15.524 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.e 30/160 2024-09-05T01:04:15.535 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noar 31/160 2024-09-05T01:04:15.535 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.no 32/160 2024-09-05T01:04:15.535 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8 33/160 2024-09-05T01:04:15.535 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9 34/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 35/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el 36/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 37/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 38/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 39/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T01:04:15.536 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T01:04:15.537 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T01:04:15.538 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T01:04:15.539 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T01:04:15.540 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T01:04:15.541 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T01:04:15.542 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T01:04:15.543 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T01:04:15.544 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 149/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 151/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T01:04:15.545 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T01:04:18.075 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T01:04:18.076 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.077 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.078 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.1.1-259.g91c8a7e5.el9.noarch 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T01:04:18.079 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.080 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T01:04:18.081 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T01:04:18.082 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T01:04:18.083 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T01:04:18.084 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T01:04:18.085 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T01:04:18.086 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T01:04:18.087 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.1.1-259.g91c8a7e5.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T01:04:18.088 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T01:04:18.089 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T01:04:18.089 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:04:18.089 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-05T01:04:18.491 DEBUG:teuthology.parallel:result is None 2024-09-05T01:04:18.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:18.642 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:04:18.664 INFO:teuthology.orchestra.run.smithi129.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:18.665 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:18.665 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:04:18.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:18.809 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:04:18.835 INFO:teuthology.orchestra.run.smithi183.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:18.835 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:18.835 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:04:18.837 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:18.976 DEBUG:teuthology.orchestra.run.smithi194:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T01:04:19.002 INFO:teuthology.orchestra.run.smithi194.stdout:19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:19.002 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-259.g91c8a7e5.el9 2024-09-05T01:04:19.002 INFO:teuthology.task.install:The correct ceph version 19.1.1-259.g91c8a7e5 is installed. 2024-09-05T01:04:19.004 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T01:04:19.004 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:04:19.005 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:04:19.032 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:04:19.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:04:19.065 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:04:19.065 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T01:04:19.098 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T01:04:19.098 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:04:19.098 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:04:19.124 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:04:19.190 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:04:19.190 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:04:19.220 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:04:19.288 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:04:19.288 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T01:04:19.318 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T01:04:19.387 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T01:04:19.388 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:04:19.388 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:04:19.413 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:04:19.478 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:04:19.479 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:04:19.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:04:19.577 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:04:19.578 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T01:04:19.606 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T01:04:19.674 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T01:04:19.675 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:04:19.675 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:04:19.701 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:04:19.767 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:04:19.767 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:04:19.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:04:19.865 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:04:19.866 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T01:04:19.894 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T01:04:19.963 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-05T01:04:20.067 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T01:04:20.067 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:20.068 INFO:tasks.cephadm:Cluster fsid is c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:04:20.068 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-05T01:04:20.068 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.129', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.194'} 2024-09-05T01:04:20.068 INFO:tasks.cephadm:First mon is mon.a on smithi129 2024-09-05T01:04:20.069 INFO:tasks.cephadm:First mgr is x 2024-09-05T01:04:20.069 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-05T01:04:20.069 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-09-05T01:04:20.097 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-09-05T01:04:20.127 DEBUG:teuthology.orchestra.run.smithi194:> sudo hostname $(hostname -s) 2024-09-05T01:04:20.156 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-05T01:04:20.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:20.293 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56000/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-259-g91c8a7e5', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-259.g91c8a7e5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-04 16:53:37.418684', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-05T01:04:20.433 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:04:20.435 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T01:04:20.435 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm 2024-09-05T01:04:20.435 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:20.865 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:04 /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:20.865 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.255 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:04 /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.256 DEBUG:teuthology.orchestra.run.smithi194:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.648 INFO:teuthology.orchestra.run.smithi194.stdout:-rw-r--r--. 1 ubuntu ubuntu 784521 Sep 5 01:04 /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.648 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.668 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.687 DEBUG:teuthology.orchestra.run.smithi194:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T01:04:21.723 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 on all hosts... 2024-09-05T01:04:21.723 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:04:21.726 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:04:21.730 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 pull 2024-09-05T01:04:23.206 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:04:23.350 INFO:teuthology.orchestra.run.smithi194.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:04:23.363 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:04:49.750 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:04:49.750 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:04:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:04:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-09-05T01:04:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:04:49.751 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:04:49.751 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:05:06.705 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-09-05T01:05:06.706 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-05T01:05:13.162 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "image_id": "77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac", 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "repo_digests": [ 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067" 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout: ] 2024-09-05T01:05:13.163 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-09-05T01:05:13.188 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-09-05T01:05:13.222 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-09-05T01:05:13.257 DEBUG:teuthology.orchestra.run.smithi194:> sudo mkdir -p /etc/ceph 2024-09-05T01:05:13.284 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-09-05T01:05:13.313 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-09-05T01:05:13.343 DEBUG:teuthology.orchestra.run.smithi194:> sudo chmod 777 /etc/ceph 2024-09-05T01:05:13.368 INFO:tasks.cephadm:Writing seed config... 2024-09-05T01:05:13.369 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-05T01:05:13.370 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-05T01:05:13.371 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-05T01:05:13.372 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-05T01:05:13.373 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-05T01:05:13.374 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:05:13.374 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-05T01:05:13.393 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c8062bdc-6b22-11ef-bcd6-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-05T01:05:13.394 DEBUG:teuthology.orchestra.run.smithi129:mon.a> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service 2024-09-05T01:05:13.436 DEBUG:teuthology.orchestra.run.smithi129:mgr.x> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service 2024-09-05T01:05:13.478 INFO:tasks.cephadm:Bootstrapping... 2024-09-05T01:05:13.479 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 -v bootstrap --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.129 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:05:13.688 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-05T01:05:13.688 INFO:teuthology.orchestra.run.smithi129.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4', '-v', 'bootstrap', '--fsid', 'c8062bdc-6b22-11ef-bcd6-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.129', '--skip-admin-label'] 2024-09-05T01:05:13.688 INFO:teuthology.orchestra.run.smithi129.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-05T01:05:13.688 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying podman|docker is present... 2024-09-05T01:05:13.712 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 5.2.2 2024-09-05T01:05:13.712 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying lvm2 is present... 2024-09-05T01:05:13.713 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying time synchronization is in place... 2024-09-05T01:05:13.720 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T01:05:13.720 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T01:05:13.727 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T01:05:13.727 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:05:13.733 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout enabled 2024-09-05T01:05:13.740 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout active 2024-09-05T01:05:13.740 INFO:teuthology.orchestra.run.smithi129.stdout:Unit chronyd.service is enabled and running 2024-09-05T01:05:13.740 INFO:teuthology.orchestra.run.smithi129.stdout:Repeating the final host check... 2024-09-05T01:05:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 5.2.2 2024-09-05T01:05:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-05T01:05:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl is present 2024-09-05T01:05:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:lvcreate is present 2024-09-05T01:05:13.771 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T01:05:13.771 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T01:05:13.777 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T01:05:13.777 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:05:13.784 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout enabled 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout active 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Unit chronyd.service is enabled and running 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Host looks OK 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Cluster fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Acquiring lock 140025151144960 on /run/cephadm/c8062bdc-6b22-11ef-bcd6-c7b262605968.lock 2024-09-05T01:05:13.790 INFO:teuthology.orchestra.run.smithi129.stdout:Lock 140025151144960 acquired on /run/cephadm/c8062bdc-6b22-11ef-bcd6-c7b262605968.lock 2024-09-05T01:05:13.791 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying IP 172.21.15.129 port 3300 ... 2024-09-05T01:05:13.791 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying IP 172.21.15.129 port 6789 ... 2024-09-05T01:05:13.791 INFO:teuthology.orchestra.run.smithi129.stdout:Base mon IP(s) is [172.21.15.129:3300, 172.21.15.129:6789], mon addrv is [v2:172.21.15.129:3300,v1:172.21.15.129:6789] 2024-09-05T01:05:13.794 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.129 metric 100 2024-09-05T01:05:13.794 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.129 metric 100 2024-09-05T01:05:13.797 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-05T01:05:13.797 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-05T01:05:13.797 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-05T01:05:13.799 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-05T01:05:13.799 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-05T01:05:13.800 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T01:05:13.800 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-05T01:05:13.800 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7261/64 scope link noprefixroute 2024-09-05T01:05:13.800 INFO:teuthology.orchestra.run.smithi129.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T01:05:13.800 INFO:teuthology.orchestra.run.smithi129.stdout:Mon IP `172.21.15.129` is in CIDR network `172.21.0.0/20` 2024-09-05T01:05:13.801 INFO:teuthology.orchestra.run.smithi129.stdout:Mon IP `172.21.15.129` is in CIDR network `172.21.0.0/20` 2024-09-05T01:05:13.801 INFO:teuthology.orchestra.run.smithi129.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-05T01:05:13.801 INFO:teuthology.orchestra.run.smithi129.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-05T01:05:13.802 INFO:teuthology.orchestra.run.smithi129.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stdout 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4... 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Getting image source signatures 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying blob sha256:3f43494a24a2d1fe0d791eb19b05391e70ab9f54611bc9deae80cf7c3fb2dba9 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-05T01:05:17.231 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Copying config sha256:77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac 2024-09-05T01:05:17.232 INFO:teuthology.orchestra.run.smithi129.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-05T01:05:25.432 INFO:teuthology.orchestra.run.smithi129.stdout:ceph: stdout ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T01:05:25.432 INFO:teuthology.orchestra.run.smithi129.stdout:Ceph version: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable) 2024-09-05T01:05:25.432 INFO:teuthology.orchestra.run.smithi129.stdout:Extracting ceph user uid/gid from container image... 2024-09-05T01:05:26.385 INFO:teuthology.orchestra.run.smithi129.stdout:stat: stdout 167 167 2024-09-05T01:05:26.385 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial keys... 2024-09-05T01:05:27.274 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQDWA9lmi9ovKxAAPgJy9DuTNX4gQuHaf8C/Tw== 2024-09-05T01:05:28.204 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQDXA9lmQOsyIxAA8AOaCcZIt0BkFawX9d1h5Q== 2024-09-05T01:05:29.227 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph-authtool: stdout AQDYA9lmwbIUIRAAH0dlCl72zgPew6/EaqkJOQ== 2024-09-05T01:05:29.227 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial monmap... 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:monmaptool for a [v2:172.21.15.129:3300,v1:172.21.15.129:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:setting min_mon_release = quincy 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: set fsid to c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:30.324 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T01:05:30.325 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:05:30.325 INFO:teuthology.orchestra.run.smithi129.stdout:Creating mon... 2024-09-05T01:05:31.329 INFO:teuthology.orchestra.run.smithi129.stdout:create mon.a on 2024-09-05T01:05:31.530 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-05T01:05:31.685 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-05T01:05:31.872 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target. 2024-09-05T01:05:31.873 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target. 2024-09-05T01:05:32.115 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a 2024-09-05T01:05:32.115 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service: Unit ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service not loaded. 2024-09-05T01:05:32.288 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target.wants/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service → /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@.service. 2024-09-05T01:05:32.773 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:05:32.773 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-09-05T01:05:32.780 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:05:32.780 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:05:32.780 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-09-05T01:05:32.780 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T01:05:32.781 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mon to start... 2024-09-05T01:05:32.781 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mon... 2024-09-05T01:05:33.838 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout cluster: 2024-09-05T01:05:33.838 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout id: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:33.838 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-05T01:05:33.838 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:33.838 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout services: 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.524006s) 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout data: 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout pgs: 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb22ef3640 1 Processor -- start 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb22ef3640 1 -- start start 2024-09-05T01:05:33.839 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb22ef3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb22ef3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c07c850 con 0x7fdb1c07de20 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb20c8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb20c8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44708/0 (socket says 172.21.15.129:44708) 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.194+0000 7fdb20c8f640 1 -- 172.21.15.129:0/2155453081 learned_addr learned my addr 172.21.15.129:0/2155453081 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.195+0000 7fdb20c8f640 1 -- 172.21.15.129:0/2155453081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c07c9d0 con 0x7fdb1c07de20 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.196+0000 7fdb20c8f640 1 --2- 172.21.15.129:0/2155453081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009e80 tx=0x7fdb1402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552378cfc314ba0e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb1b7fe640 1 -- 172.21.15.129:0/2155453081 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb14036070 con 0x7fdb1c07de20 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb1b7fe640 1 -- 172.21.15.129:0/2155453081 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb14031400 con 0x7fdb1c07de20 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb1b7fe640 1 -- 172.21.15.129:0/2155453081 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb14031700 con 0x7fdb1c07de20 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 -- 172.21.15.129:0/2155453081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 msgr2=0x7fdb1c07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:33.840 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 --2- 172.21.15.129:0/2155453081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009e80 tx=0x7fdb1402a7a0 comp rx=0 tx=0).stop 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 -- 172.21.15.129:0/2155453081 shutdown_connections 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 --2- 172.21.15.129:0/2155453081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c07c280 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 -- 172.21.15.129:0/2155453081 >> 172.21.15.129:0/2155453081 conn(0x7fdb1c102da0 msgr2=0x7fdb1c1051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 -- 172.21.15.129:0/2155453081 shutdown_connections 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.197+0000 7fdb22ef3640 1 -- 172.21.15.129:0/2155453081 wait complete. 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb22ef3640 1 Processor -- start 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb22ef3640 1 -- start start 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb22ef3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb22ef3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c07cf00 con 0x7fdb1c07de20 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb20c8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb20c8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44710/0 (socket says 172.21.15.129:44710) 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.198+0000 7fdb20c8f640 1 -- 172.21.15.129:0/1302551821 learned_addr learned my addr 172.21.15.129:0/1302551821 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:33.841 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb20c8f640 1 -- 172.21.15.129:0/1302551821 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c19d340 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb20c8f640 1 --2- 172.21.15.129:0/1302551821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb14034040 tx=0x7fdb14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb14041020 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb1403c660 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c19d5d0 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb14036040 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a02c0 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb14044020 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.199+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdb1403d940 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.200+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb1c07c300 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.202+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdb1404b030 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.261+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fdb1c1a0820 con 0x7fdb1c07de20 2024-09-05T01:05:33.842 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.261+0000 7fdb19ffb640 1 -- 172.21.15.129:0/1302551821 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fdb1403d420 con 0x7fdb1c07de20 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 msgr2=0x7fdb1c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 --2- 172.21.15.129:0/1302551821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdb14034040 tx=0x7fdb14004a40 comp rx=0 tx=0).stop 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 shutdown_connections 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 --2- 172.21.15.129:0/1302551821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb1c07de20 0x7fdb1c19ce00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 >> 172.21.15.129:0/1302551821 conn(0x7fdb1c102da0 msgr2=0x7fdb1c103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 shutdown_connections 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:33.262+0000 7fdb22ef3640 1 -- 172.21.15.129:0/1302551821 wait complete. 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:mon is available 2024-09-05T01:05:33.843 INFO:teuthology.orchestra.run.smithi129.stdout:Assimilating anything we can from ceph.conf... 2024-09-05T01:05:34.911 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:34 smithi129 ceph-mon[30613]: from='client.? 172.21.15.129:0/2317925776' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-05T01:05:34.919 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:34.919 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout fsid = c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.129:3300,v1:172.21.15.129:6789] 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T01:05:34.920 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.243+0000 7f61a2905640 1 Processor -- start 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.243+0000 7f61a2905640 1 -- start start 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f61a2905640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f61a2905640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f619c106e70 con 0x7f619c1064a0 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f619bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f619bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44712/0 (socket says 172.21.15.129:44712) 2024-09-05T01:05:34.921 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f619bfff640 1 -- 172.21.15.129:0/799092941 learned_addr learned my addr 172.21.15.129:0/799092941 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.244+0000 7f619bfff640 1 -- 172.21.15.129:0/799092941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c1076a0 con 0x7f619c1064a0 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f619bfff640 1 --2- 172.21.15.129:0/799092941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f618c009820 tx=0x7f618c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24ac6d6ad5ff12a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f619b7fe640 1 -- 172.21.15.129:0/799092941 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c037070 con 0x7f619c1064a0 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f619b7fe640 1 -- 172.21.15.129:0/799092941 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c032440 con 0x7f619c1064a0 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f619b7fe640 1 -- 172.21.15.129:0/799092941 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c032740 con 0x7f619c1064a0 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 -- 172.21.15.129:0/799092941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 msgr2=0x7f619c1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 --2- 172.21.15.129:0/799092941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f618c009820 tx=0x7f618c02a270 comp rx=0 tx=0).stop 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 -- 172.21.15.129:0/799092941 shutdown_connections 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 --2- 172.21.15.129:0/799092941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c1068a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 -- 172.21.15.129:0/799092941 >> 172.21.15.129:0/799092941 conn(0x7f619c101c90 msgr2=0x7f619c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 -- 172.21.15.129:0/799092941 shutdown_connections 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.245+0000 7f61a2905640 1 -- 172.21.15.129:0/799092941 wait complete. 2024-09-05T01:05:34.922 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f61a2905640 1 Processor -- start 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f61a2905640 1 -- start start 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f61a2905640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f61a2905640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f619c107c40 con 0x7f619c1064a0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f619bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f619bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44726/0 (socket says 172.21.15.129:44726) 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f619bfff640 1 -- 172.21.15.129:0/2317925776 learned_addr learned my addr 172.21.15.129:0/2317925776 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.246+0000 7f619bfff640 1 -- 172.21.15.129:0/2317925776 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c18f5a0 con 0x7f619c1064a0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f619bfff640 1 --2- 172.21.15.129:0/2317925776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f618c0090b0 tx=0x7f618c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c041070 con 0x7f619c1064a0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f618c03c7a0 con 0x7f619c1064a0 2024-09-05T01:05:34.923 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c18d790 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f618c037040 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c18dcd0 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f618c044070 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.247+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f618c03d3f0 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.248+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f619c108290 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.250+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f618c046450 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.306+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f619c18df50 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.309+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f618c046630 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.310+0000 7f6199ffb640 1 -- 172.21.15.129:0/2317925776 <== mon.0 v2:172.21.15.129:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f618c0584b0 con 0x7f619c1064a0 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 msgr2=0x7f619c18f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:34.924 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 --2- 172.21.15.129:0/2317925776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f618c0090b0 tx=0x7f618c0046c0 comp rx=0 tx=0).stop 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 shutdown_connections 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 --2- 172.21.15.129:0/2317925776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f619c1064a0 0x7f619c18f060 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 >> 172.21.15.129:0/2317925776 conn(0x7f619c101c90 msgr2=0x7f619c07b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 shutdown_connections 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:34.311+0000 7f61a2905640 1 -- 172.21.15.129:0/2317925776 wait complete. 2024-09-05T01:05:34.925 INFO:teuthology.orchestra.run.smithi129.stdout:Generating new minimal ceph.conf... 2024-09-05T01:05:35.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:35 smithi129 ceph-mon[30613]: from='client.? 172.21.15.129:0/4108148435' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829fe49640 1 Processor -- start 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829fe49640 1 -- start start 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829fe49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829fe49640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f829807bd90 con 0x7f82981080d0 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829dbe5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829dbe5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44732/0 (socket says 172.21.15.129:44732) 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.531+0000 7f829dbe5640 1 -- 172.21.15.129:0/1822659668 learned_addr learned my addr 172.21.15.129:0/1822659668 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:36.175 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.532+0000 7f829dbe5640 1 -- 172.21.15.129:0/1822659668 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829810aa00 con 0x7f82981080d0 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.532+0000 7f829dbe5640 1 --2- 172.21.15.129:0/1822659668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8294009820 tx=0x7f829402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2889d495bfa0a098 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.532+0000 7f829cbe3640 1 -- 172.21.15.129:0/1822659668 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8294037070 con 0x7f82981080d0 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.532+0000 7f829cbe3640 1 -- 172.21.15.129:0/1822659668 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f82940328d0 con 0x7f82981080d0 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.532+0000 7f829cbe3640 1 -- 172.21.15.129:0/1822659668 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82940307d0 con 0x7f82981080d0 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 -- 172.21.15.129:0/1822659668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 msgr2=0x7f829810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 --2- 172.21.15.129:0/1822659668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8294009820 tx=0x7f829402a270 comp rx=0 tx=0).stop 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 -- 172.21.15.129:0/1822659668 shutdown_connections 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 --2- 172.21.15.129:0/1822659668 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829810a4c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 -- 172.21.15.129:0/1822659668 >> 172.21.15.129:0/1822659668 conn(0x7f8298101c90 msgr2=0x7f82981040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:36.176 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 -- 172.21.15.129:0/1822659668 shutdown_connections 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.533+0000 7f829fe49640 1 -- 172.21.15.129:0/1822659668 wait complete. 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829fe49640 1 Processor -- start 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829fe49640 1 -- start start 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829fe49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829fe49640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f829810b980 con 0x7f82981080d0 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829dbe5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829dbe5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44740/0 (socket says 172.21.15.129:44740) 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829dbe5640 1 -- 172.21.15.129:0/4108148435 learned_addr learned my addr 172.21.15.129:0/4108148435 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829dbe5640 1 -- 172.21.15.129:0/4108148435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829819b950 con 0x7f82981080d0 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.534+0000 7f829dbe5640 1 --2- 172.21.15.129:0/4108148435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82940097f0 tx=0x7f8294030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:36.177 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8294037040 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8294032bb0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f829403a390 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829819bbe0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829819e8d0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f829403a5d0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.535+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8294044bf0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.536+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8298108530 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.538+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8294044400 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.595+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f829819eda0 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.595+0000 7f828effd640 1 -- 172.21.15.129:0/4108148435 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f8294032d60 con 0x7f82981080d0 2024-09-05T01:05:36.178 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 msgr2=0x7f829819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 --2- 172.21.15.129:0/4108148435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82940097f0 tx=0x7f8294030d00 comp rx=0 tx=0).stop 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 shutdown_connections 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 --2- 172.21.15.129:0/4108148435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f82981080d0 0x7f829819b410 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 >> 172.21.15.129:0/4108148435 conn(0x7f8298101c90 msgr2=0x7f8298102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 shutdown_connections 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:35.596+0000 7f829fe49640 1 -- 172.21.15.129:0/4108148435 wait complete. 2024-09-05T01:05:36.179 INFO:teuthology.orchestra.run.smithi129.stdout:Restarting the monitor... 2024-09-05T01:05:36.446 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 systemd[1]: Stopping Ceph mon.a for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:05:36.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a[30609]: 2024-09-05T01:05:36.445+0000 7f44e236c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:05:36.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a[30609]: 2024-09-05T01:05:36.445+0000 7f44e236c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-05T01:05:36.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 podman[30856]: 2024-09-05 01:05:36.629332038 +0000 UTC m=+0.279503715 container died 6f9e74b8b666f4d6ce56d82cf8de3612bbc3a366a529ff159a7b2572b7b78f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, ceph=True) 2024-09-05T01:05:37.146 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 podman[30856]: 2024-09-05 01:05:36.911448685 +0000 UTC m=+0.561620363 container cleanup 6f9e74b8b666f4d6ce56d82cf8de3612bbc3a366a529ff159a7b2572b7b78f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-05T01:05:37.146 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:36 smithi129 bash[30856]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a 2024-09-05T01:05:37.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 podman[30870]: 2024-09-05 01:05:37.145559449 +0000 UTC m=+0.519023675 container remove 6f9e74b8b666f4d6ce56d82cf8de3612bbc3a366a529ff159a7b2572b7b78f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T01:05:37.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-05T01:05:37.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 systemd[1]: Stopped Ceph mon.a for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:05:37.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 systemd[1]: Starting Ceph mon.a for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:05:37.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 podman[30960]: 2024-09-05 01:05:37.393515349 +0000 UTC m=+0.015651508 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 podman[30960]: 2024-09-05 01:05:37.496426237 +0000 UTC m=+0.118562429 container create ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 podman[30960]: 2024-09-05 01:05:37.682378101 +0000 UTC m=+0.304514247 container init ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e) 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 podman[30960]: 2024-09-05 01:05:37.685498728 +0000 UTC m=+0.307634875 container start ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable), process ceph-mon, pid 2 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: pidfile_write: ignore empty --pid-file 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: load: jerasure load: lrc 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: RocksDB version: 7.9.2 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Git sha 0 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Compile date 2024-09-04 15:32:03 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: DB SUMMARY 2024-09-05T01:05:37.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: DB Session ID: GEZZ0YPNF6WJFIP5LFMB 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: CURRENT file: CURRENT 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104870 ; 2024-09-05T01:05:37.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.error_if_exists: 0 2024-09-05T01:05:37.745 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.create_if_missing: 0 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.paranoid_checks: 1 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.env: 0x5579ff227940 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.info_log: 0x557a00c036e0 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.statistics: (nil) 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.use_fsync: 0 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_log_file_size: 0 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T01:05:37.746 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_fallocate: 1 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.use_direct_reads: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.db_log_dir: 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.wal_dir: 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T01:05:37.747 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.write_buffer_manager: 0x557a00c12dc0 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T01:05:37.748 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.unordered_write: 0 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.row_cache: None 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.wal_filter: None 2024-09-05T01:05:37.749 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.two_write_queues: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.wal_compression: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.atomic_flush: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.log_readahead_size: 0 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T01:05:37.752 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_background_jobs: 2 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_background_compactions: -1 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_subcompactions: 1 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T01:05:37.753 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_open_files: -1 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_background_flushes: -1 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Compression algorithms supported: 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kZSTD supported: 0 2024-09-05T01:05:37.754 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kXpressCompression supported: 0 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kBZip2Compression supported: 0 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kLZ4Compression supported: 1 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kZlibCompression supported: 1 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: kSnappyCompression supported: 1 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T01:05:37.755 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.merge_operator: 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_filter: None 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557a00c03320) 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_type: 0 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: data_block_index_type: 0 2024-09-05T01:05:37.756 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_shortening: 1 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: checksum: 4 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: no_block_cache: 0 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache: 0x557a00bf71f0 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_options: 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: capacity : 536870912 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: num_shard_bits : 4 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: strict_capacity_limit : 0 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_cache_compressed: (nil) 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: persistent_cache: (nil) 2024-09-05T01:05:37.757 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_size: 4096 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_size_deviation: 10 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_restart_interval: 16 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: index_block_restart_interval: 1 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: metadata_block_size: 4096 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: partition_filters: 0 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: use_delta_encoding: 1 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: filter_policy: bloomfilter 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: whole_key_filtering: 1 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: verify_compression: 0 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: format_version: 5 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: enable_index_compression: 1 2024-09-05T01:05:37.758 INFO:journalctl@ceph.mon.a.smithi129.stdout: block_align: 0 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout: max_auto_readahead_size: 262144 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout: prepopulate_block_cache: 0 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout: initial_auto_readahead_size: 8192 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression: NoCompression 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.num_levels: 7 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T01:05:37.759 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T01:05:37.760 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T01:05:37.761 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T01:05:37.762 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T01:05:37.763 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.inplace_update_support: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.bloom_locality: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.max_successive_merges: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T01:05:37.764 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.ttl: 2592000 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enable_blob_files: false 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.min_blob_size: 0 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T01:05:37.765 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f16cc66e-e496-4b3a-be5f-69c2fb019add 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725498337712656, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725498337713690, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97899, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14187, "raw_average_key_size": 49, "raw_value_size": 90188, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725498337, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f16cc66e-e496-4b3a-be5f-69c2fb019add", "db_session_id": "GEZZ0YPNF6WJFIP5LFMB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725498337713795, "job": 1, "event": "recovery_finished"} 2024-09-05T01:05:37.766 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557a00c24e00 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: DB pointer 0x557a00d20000 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** DB Stats ** 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T01:05:37.767 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: L0 2/0 99.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: Sum 2/0 99.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T01:05:37.768 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Cumulative compaction: 0.00 GB write, 30.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Interval compaction: 0.00 GB write, 30.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T01:05:37.769 INFO:journalctl@ceph.mon.a.smithi129.stdout: Block cache BinnedLRUCache@0x557a00bf71f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: starting mon.a rank 0 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???) e1 preinit fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).mds e1 new map 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).mds e1 print_map 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: e1 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: btime 2024-09-05T01:05:32:738936+0000 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: legacy client fscid: -1 2024-09-05T01:05:37.770 INFO:journalctl@ceph.mon.a.smithi129.stdout: 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout: No filesystems configured 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 bash[30960]: ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e 2024-09-05T01:05:37.771 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 systemd[1]: Started Ceph mon.a for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:05:37.771 INFO:teuthology.orchestra.run.smithi129.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: monmap epoch 1 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: last_changed 2024-09-05T01:05:29.598613+0000 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: election_strategy: 1 2024-09-05T01:05:38.060 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:05:38.061 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: fsmap 2024-09-05T01:05:38.061 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T01:05:38.061 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:37 smithi129 ceph-mon[30979]: mgrmap e1: no daemons active 2024-09-05T01:05:38.780 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.140+0000 7fe9a46de640 1 Processor -- start 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.140+0000 7fe9a46de640 1 -- start start 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a46de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a46de640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c106d90 con 0x7fe99c1063c0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a247a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a247a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44756/0 (socket says 172.21.15.129:44756) 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a247a640 1 -- 172.21.15.129:0/1353387749 learned_addr learned my addr 172.21.15.129:0/1353387749 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.141+0000 7fe9a247a640 1 -- 172.21.15.129:0/1353387749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c1075c0 con 0x7fe99c1063c0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a247a640 1 --2- 172.21.15.129:0/1353387749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe994009820 tx=0x7fe99402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3026e2c9d17f6e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a1478640 1 -- 172.21.15.129:0/1353387749 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe994037070 con 0x7fe99c1063c0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a1478640 1 -- 172.21.15.129:0/1353387749 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe9940328d0 con 0x7fe99c1063c0 2024-09-05T01:05:38.781 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a1478640 1 -- 172.21.15.129:0/1353387749 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9940307d0 con 0x7fe99c1063c0 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 -- 172.21.15.129:0/1353387749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 msgr2=0x7fe99c1067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 --2- 172.21.15.129:0/1353387749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe994009820 tx=0x7fe99402a270 comp rx=0 tx=0).stop 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 -- 172.21.15.129:0/1353387749 shutdown_connections 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 --2- 172.21.15.129:0/1353387749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c1067c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 -- 172.21.15.129:0/1353387749 >> 172.21.15.129:0/1353387749 conn(0x7fe99c101b30 msgr2=0x7fe99c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 -- 172.21.15.129:0/1353387749 shutdown_connections 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.144+0000 7fe9a46de640 1 -- 172.21.15.129:0/1353387749 wait complete. 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a46de640 1 Processor -- start 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a46de640 1 -- start start 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a46de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a46de640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c074a20 con 0x7fe99c1063c0 2024-09-05T01:05:38.782 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a247a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a247a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44770/0 (socket says 172.21.15.129:44770) 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.145+0000 7fe9a247a640 1 -- 172.21.15.129:0/1345924889 learned_addr learned my addr 172.21.15.129:0/1345924889 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe9a247a640 1 -- 172.21.15.129:0/1345924889 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c1996b0 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe9a247a640 1 --2- 172.21.15.129:0/1345924889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe994009940 tx=0x7fe994032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe994042070 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe994030c80 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c199940 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe994037040 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c19c630 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe994045070 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.146+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe994047da0 con 0x7fe99c1063c0 2024-09-05T01:05:38.783 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.147+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c1067c0 con 0x7fe99c1063c0 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.149+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe994047440 con 0x7fe99c1063c0 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.206+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fe99c192630 con 0x7fe99c1063c0 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe99b7fe640 1 -- 172.21.15.129:0/1345924889 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fe99403c780 con 0x7fe99c1063c0 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 msgr2=0x7fe99c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 --2- 172.21.15.129:0/1345924889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe994009940 tx=0x7fe994032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 shutdown_connections 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 --2- 172.21.15.129:0/1345924889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe99c1063c0 0x7fe99c199170 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 >> 172.21.15.129:0/1345924889 conn(0x7fe99c101b30 msgr2=0x7fe99c18f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 shutdown_connections 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:38.211+0000 7fe9a46de640 1 -- 172.21.15.129:0/1345924889 wait complete. 2024-09-05T01:05:38.784 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-05T01:05:38.805 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:05:38.805 INFO:teuthology.orchestra.run.smithi129.stdout:Creating mgr... 2024-09-05T01:05:38.805 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-05T01:05:38.806 INFO:teuthology.orchestra.run.smithi129.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-05T01:05:39.064 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T01:05:39.064 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service: Unit ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service not loaded. 2024-09-05T01:05:39.219 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968.target.wants/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@.service. 2024-09-05T01:05:39.451 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1345924889' entity='client.admin' 2024-09-05T01:05:39.451 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 systemd[1]: Starting Ceph mgr.x for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:05:39.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 podman[31189]: 2024-09-05 01:05:39.353053865 +0000 UTC m=+0.015336849 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 2024-09-05T01:05:39.743 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 podman[31189]: 2024-09-05 01:05:39.45694448 +0000 UTC m=+0.119227454 container create 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-05T01:05:39.743 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 podman[31189]: 2024-09-05 01:05:39.663046065 +0000 UTC m=+0.325329038 container init 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T01:05:39.743 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 podman[31189]: 2024-09-05 01:05:39.665921268 +0000 UTC m=+0.328204242 container start 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T01:05:39.791 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:05:39.791 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-09-05T01:05:39.800 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:05:39.801 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:05:39.801 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-09-05T01:05:39.801 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T01:05:39.808 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:05:39.808 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr to start... 2024-09-05T01:05:39.816 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr... 2024-09-05T01:05:40.070 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 bash[31189]: 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 2024-09-05T01:05:40.071 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 systemd[1]: Started Ceph mgr.x for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:05:40.071 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:39.787+0000 7f29bc0c9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:05:40.071 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:39.850+0000 7f29bc0c9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:05:40.071 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.001+0000 7f29bc0c9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:05:40.071 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.070+0000 7f29bc0c9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:05:40.321 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.222+0000 7f29bc0c9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:05:40.690 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3157476600' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:05:40.691 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.373+0000 7f29bc0c9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:05:40.691 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.689+0000 7f29bc0c9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:05:40.876 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:40.886 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:05:40.886 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:05:40.886 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:40.887 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:05:40.888 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.889 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:05:32:738936+0000", 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:05:40.890 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:05:32.739727+0000", 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.249+0000 7efe3b9cc640 1 Processor -- start 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.249+0000 7efe3b9cc640 1 -- start start 2024-09-05T01:05:40.891 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe3b9cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe3b9cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe341061a0 con 0x7efe341057d0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe39768640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe39768640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44796/0 (socket says 172.21.15.129:44796) 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe39768640 1 -- 172.21.15.129:0/2601800887 learned_addr learned my addr 172.21.15.129:0/2601800887 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.250+0000 7efe39768640 1 -- 172.21.15.129:0/2601800887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe34106320 con 0x7efe341057d0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe39768640 1 --2- 172.21.15.129:0/2601800887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe30009820 tx=0x7efe3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ac533bde68192d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe2bfff640 1 -- 172.21.15.129:0/2601800887 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30037070 con 0x7efe341057d0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe2bfff640 1 -- 172.21.15.129:0/2601800887 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efe300328d0 con 0x7efe341057d0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe2bfff640 1 -- 172.21.15.129:0/2601800887 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe300307d0 con 0x7efe341057d0 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe3b9cc640 1 -- 172.21.15.129:0/2601800887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 msgr2=0x7efe34105bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe3b9cc640 1 --2- 172.21.15.129:0/2601800887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe30009820 tx=0x7efe3002a270 comp rx=0 tx=0).stop 2024-09-05T01:05:40.892 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe3b9cc640 1 -- 172.21.15.129:0/2601800887 shutdown_connections 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.252+0000 7efe3b9cc640 1 --2- 172.21.15.129:0/2601800887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe34105bd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 -- 172.21.15.129:0/2601800887 >> 172.21.15.129:0/2601800887 conn(0x7efe34101300 msgr2=0x7efe34103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 -- 172.21.15.129:0/2601800887 shutdown_connections 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 -- 172.21.15.129:0/2601800887 wait complete. 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 Processor -- start 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 -- start start 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.253+0000 7efe3b9cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe34109650 con 0x7efe341057d0 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe39768640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe39768640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:44808/0 (socket says 172.21.15.129:44808) 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe39768640 1 -- 172.21.15.129:0/3157476600 learned_addr learned my addr 172.21.15.129:0/3157476600 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:40.893 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe39768640 1 -- 172.21.15.129:0/3157476600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3419b8c0 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe39768640 1 --2- 172.21.15.129:0/3157476600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe30009940 tx=0x7efe30032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30042070 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efe30030c80 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe30037040 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3419bbb0 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.254+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3419e840 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.255+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe30045070 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.255+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe30047a00 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.255+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe34105c50 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.257+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7efe3003c700 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.318+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7efe3419ea10 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.318+0000 7efe2a7fc640 1 -- 172.21.15.129:0/3157476600 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7efe3003c8e0 con 0x7efe341057d0 2024-09-05T01:05:40.894 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 msgr2=0x7efe3419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 --2- 172.21.15.129:0/3157476600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe30009940 tx=0x7efe30032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 shutdown_connections 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 --2- 172.21.15.129:0/3157476600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe341057d0 0x7efe3419b380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 >> 172.21.15.129:0/3157476600 conn(0x7efe34101300 msgr2=0x7efe34101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 shutdown_connections 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:40.319+0000 7efe3b9cc640 1 -- 172.21.15.129:0/3157476600 wait complete. 2024-09-05T01:05:40.895 INFO:teuthology.orchestra.run.smithi129.stdout:mgr not available, waiting (1/15)... 2024-09-05T01:05:40.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.749+0000 7f29bc0c9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:05:40.971 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:40.936+0000 7f29bc0c9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:05:42.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:41.802+0000 7f29bc0c9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:05:42.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:41.868+0000 7f29bc0c9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:05:42.688 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:42.574+0000 7f29bc0c9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:05:42.688 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:42.687+0000 7f29bc0c9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:05:42.965 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:42.828+0000 7f29bc0c9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:05:42.965 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:42.888+0000 7f29bc0c9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:05:43.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.049+0000 7f29bc0c9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:05:43.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.117+0000 7f29bc0c9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:05:43.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.178+0000 7f29bc0c9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:05:43.563 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4103280396' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:05:43.563 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.342+0000 7f29bc0c9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:05:43.563 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.413+0000 7f29bc0c9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:05:43.563 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.481+0000 7f29bc0c9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:05:43.563 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.562+0000 7f29bc0c9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:05:43.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.624+0000 7f29bc0c9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:05:43.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:43.684+0000 7f29bc0c9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:05:44.010 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:44.010 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:05:44.010 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:05:44.010 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:05:44.010 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:44.011 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:05:44.012 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:05:44.020 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:05:44.020 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:05:44.020 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:05:44.020 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.021 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:05:32:738936+0000", 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:05:44.022 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:05:32.739727+0000", 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c163da640 1 Processor -- start 2024-09-05T01:05:44.023 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c163da640 1 -- start start 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c163da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c163da640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10075740 con 0x7f2c10107f70 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54828/0 (socket says 172.21.15.129:54828) 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c0ffff640 1 -- 172.21.15.129:0/3865735028 learned_addr learned my addr 172.21.15.129:0/3865735028 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.319+0000 7f2c0ffff640 1 -- 172.21.15.129:0/3865735028 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1010a8a0 con 0x7f2c10107f70 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.320+0000 7f2c0ffff640 1 --2- 172.21.15.129:0/3865735028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009820 tx=0x7f2c0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12fc6bf8a872afa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.320+0000 7f2c0effd640 1 -- 172.21.15.129:0/3865735028 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c00037070 con 0x7f2c10107f70 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.320+0000 7f2c0effd640 1 -- 172.21.15.129:0/3865735028 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2c000328d0 con 0x7f2c10107f70 2024-09-05T01:05:44.024 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.320+0000 7f2c0effd640 1 -- 172.21.15.129:0/3865735028 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c000307d0 con 0x7f2c10107f70 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 -- 172.21.15.129:0/3865735028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 msgr2=0x7f2c1010a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 --2- 172.21.15.129:0/3865735028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009820 tx=0x7f2c0002a270 comp rx=0 tx=0).stop 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 -- 172.21.15.129:0/3865735028 shutdown_connections 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 --2- 172.21.15.129:0/3865735028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1010a360 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 -- 172.21.15.129:0/3865735028 >> 172.21.15.129:0/3865735028 conn(0x7f2c10101b30 msgr2=0x7f2c10103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 -- 172.21.15.129:0/3865735028 shutdown_connections 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 -- 172.21.15.129:0/3865735028 wait complete. 2024-09-05T01:05:44.025 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.321+0000 7f2c163da640 1 Processor -- start 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c163da640 1 -- start start 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c163da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c163da640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c1010b880 con 0x7f2c10107f70 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54832/0 (socket says 172.21.15.129:54832) 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0ffff640 1 -- 172.21.15.129:0/4103280396 learned_addr learned my addr 172.21.15.129:0/4103280396 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0ffff640 1 -- 172.21.15.129:0/4103280396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019b8a0 con 0x7f2c10107f70 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0ffff640 1 --2- 172.21.15.129:0/4103280396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c00032e10 tx=0x7f2c00032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.322+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c00042070 con 0x7f2c10107f70 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2c00030c80 con 0x7f2c10107f70 2024-09-05T01:05:44.026 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c163da640 1 -- 172.21.15.129:0/4103280396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019bb30 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c00037040 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c163da640 1 -- 172.21.15.129:0/4103280396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c1019e820 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c00045070 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.323+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2c000463b0 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.324+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf0005290 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.326+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2c00047440 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.386+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f2bf0005850 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.387+0000 7f2c0d7fa640 1 -- 172.21.15.129:0/4103280396 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f2c00047620 con 0x7f2c10107f70 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 msgr2=0x7f2c1019b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:44.027 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 --2- 172.21.15.129:0/4103280396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2c00032e10 tx=0x7f2c00032e40 comp rx=0 tx=0).stop 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 shutdown_connections 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 --2- 172.21.15.129:0/4103280396 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c10107f70 0x7f2c1019b360 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 >> 172.21.15.129:0/4103280396 conn(0x7f2c10101b30 msgr2=0x7f2c10102690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 shutdown_connections 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:43.388+0000 7f2bf6ffd640 1 -- 172.21.15.129:0/4103280396 wait complete. 2024-09-05T01:05:44.028 INFO:teuthology.orchestra.run.smithi129.stdout:mgr not available, waiting (2/15)... 2024-09-05T01:05:44.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:44.466+0000 7f29bc0c9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:05:44.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: Activating manager daemon x 2024-09-05T01:05:44.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: mgrmap e2: x(active, starting, since 0.0032731s) 2024-09-05T01:05:44.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:05:44.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: Manager daemon x is now available 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' 2024-09-05T01:05:44.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:44 smithi129 ceph-mon[30979]: from='mgr.14100 172.21.15.129:0/1914609668' entity='mgr.x' 2024-09-05T01:05:46.679 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:46 smithi129 ceph-mon[30979]: mgrmap e3: x(active, since 1.00674s) 2024-09-05T01:05:47.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:47.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:05:47.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:05:47.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T01:05:47.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 0 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T01:05:47.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T01:05:47.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T01:05:47.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T01:05:32:738936+0000", 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T01:05:47.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ], 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:47.305 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T01:05:32.739727+0000", 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout }, 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c9256f640 1 Processor -- start 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c9256f640 1 -- start start 2024-09-05T01:05:47.306 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c9256f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c9256f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c108440 con 0x7f6c8c107a70 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54898/0 (socket says 172.21.15.129:54898) 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c8bfff640 1 -- 172.21.15.129:0/2648676276 learned_addr learned my addr 172.21.15.129:0/2648676276 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.432+0000 7f6c8bfff640 1 -- 172.21.15.129:0/2648676276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c106680 con 0x7f6c8c107a70 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.433+0000 7f6c8bfff640 1 --2- 172.21.15.129:0/2648676276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009820 tx=0x7f6c7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63dc9634bedeffb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.433+0000 7f6c8affd640 1 -- 172.21.15.129:0/2648676276 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c037070 con 0x7f6c8c107a70 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.433+0000 7f6c8affd640 1 -- 172.21.15.129:0/2648676276 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6c7c0328d0 con 0x7f6c8c107a70 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.433+0000 7f6c8affd640 1 -- 172.21.15.129:0/2648676276 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c0307d0 con 0x7f6c8c107a70 2024-09-05T01:05:47.307 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 -- 172.21.15.129:0/2648676276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 msgr2=0x7f6c8c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 --2- 172.21.15.129:0/2648676276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c009820 tx=0x7f6c7c02a270 comp rx=0 tx=0).stop 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 -- 172.21.15.129:0/2648676276 shutdown_connections 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 --2- 172.21.15.129:0/2648676276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c107e70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 -- 172.21.15.129:0/2648676276 >> 172.21.15.129:0/2648676276 conn(0x7f6c8c101420 msgr2=0x7f6c8c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 -- 172.21.15.129:0/2648676276 shutdown_connections 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.434+0000 7f6c9256f640 1 -- 172.21.15.129:0/2648676276 wait complete. 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c9256f640 1 Processor -- start 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c9256f640 1 -- start start 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c9256f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:47.308 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c9256f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10aa40 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c8bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54908/0 (socket says 172.21.15.129:54908) 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c8bfff640 1 -- 172.21.15.129:0/1396948407 learned_addr learned my addr 172.21.15.129:0/1396948407 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.435+0000 7f6c8bfff640 1 -- 172.21.15.129:0/1396948407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c19b940 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c8bfff640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0097f0 tx=0x7f6c7c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c037040 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6c7c032bb0 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c19bbd0 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c03a390 con 0x7f6c8c107a70 2024-09-05T01:05:47.309 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.436+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c19e8c0 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f6c7c03a580 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c10abc0 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c897fa640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 0x7f6c7403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6c7c06bf20 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c8b7fe640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 0x7f6c7403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.437+0000 7f6c8b7fe640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 0x7f6c7403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c800098c0 tx=0x7f6c80006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.442+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c7c040210 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.476+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6c7c06f030 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.678+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6c8c02d8f0 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.679+0000 7f6c897fa640 1 -- 172.21.15.129:0/1396948407 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f6c7c063330 con 0x7f6c8c107a70 2024-09-05T01:05:47.310 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 msgr2=0x7f6c7403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 0x7f6c7403b3e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6c800098c0 tx=0x7f6c80006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 msgr2=0x7f6c8c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0097f0 tx=0x7f6c7c030d00 comp rx=0 tx=0).stop 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 shutdown_connections 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6c74038f20 0x7f6c7403b3e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 --2- 172.21.15.129:0/1396948407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6c8c107a70 0x7f6c8c19b400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 >> 172.21.15.129:0/1396948407 conn(0x7f6c8c101420 msgr2=0x7f6c8c192030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 shutdown_connections 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:46.681+0000 7f6c9256f640 1 -- 172.21.15.129:0/1396948407 wait complete. 2024-09-05T01:05:47.311 INFO:teuthology.orchestra.run.smithi129.stdout:mgr is available 2024-09-05T01:05:47.606 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:47 smithi129 ceph-mon[30979]: mgrmap e4: x(active, since 2s) 2024-09-05T01:05:47.606 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1396948407' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T01:05:48.451 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:48.451 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T01:05:48.451 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout fsid = c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.129:3300,v1:172.21.15.129:6789] 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T01:05:48.452 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18b59ba640 1 Processor -- start 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18b59ba640 1 -- start start 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18b59ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18b59ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0107e80 con 0x7f18b01074b0 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54918/0 (socket says 172.21.15.129:54918) 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18aeffd640 1 -- 172.21.15.129:0/287936347 learned_addr learned my addr 172.21.15.129:0/287936347 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.682+0000 7f18aeffd640 1 -- 172.21.15.129:0/287936347 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01086b0 con 0x7f18b01074b0 2024-09-05T01:05:48.453 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.683+0000 7f18aeffd640 1 --2- 172.21.15.129:0/287936347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009820 tx=0x7f18a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca15326c42f136fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.683+0000 7f18ae7fc640 1 -- 172.21.15.129:0/287936347 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0037070 con 0x7f18b01074b0 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.683+0000 7f18ae7fc640 1 -- 172.21.15.129:0/287936347 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f18a00328d0 con 0x7f18b01074b0 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.683+0000 7f18ae7fc640 1 -- 172.21.15.129:0/287936347 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a00307d0 con 0x7f18b01074b0 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- 172.21.15.129:0/287936347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 msgr2=0x7f18b01078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 --2- 172.21.15.129:0/287936347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009820 tx=0x7f18a002a270 comp rx=0 tx=0).stop 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- 172.21.15.129:0/287936347 shutdown_connections 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 --2- 172.21.15.129:0/287936347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b01078b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- 172.21.15.129:0/287936347 >> 172.21.15.129:0/287936347 conn(0x7f18b0102c60 msgr2=0x7f18b0105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- 172.21.15.129:0/287936347 shutdown_connections 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- 172.21.15.129:0/287936347 wait complete. 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 Processor -- start 2024-09-05T01:05:48.454 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.684+0000 7f18b59ba640 1 -- start start 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b59ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b59ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18b010ba60 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18aeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54930/0 (socket says 172.21.15.129:54930) 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18aeffd640 1 -- 172.21.15.129:0/1466732236 learned_addr learned my addr 172.21.15.129:0/1466732236 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18aeffd640 1 -- 172.21.15.129:0/1466732236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b019b890 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18aeffd640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009940 tx=0x7f18a0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0042070 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b019bb20 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f18a0030c80 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f18a0037040 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.685+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b019e810 con 0x7f18b01074b0 2024-09-05T01:05:48.455 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.686+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f18a0045070 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.686+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b010bbe0 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.687+0000 7f18b49b8640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 0x7f189c03b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.687+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f18a006bfc0 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.687+0000 7f18a7fff640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 0x7f189c03b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.687+0000 7f18a7fff640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 0x7f189c03b4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18980098c0 tx=0x7f1898006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.691+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18a0045360 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.857+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f18b0107930 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.858+0000 7f18b49b8640 1 -- 172.21.15.129:0/1466732236 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+431 (secure 0 0 0) 0x7f18a0050e10 con 0x7f18b01074b0 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.860+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 msgr2=0x7f189c03b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.860+0000 7f18b59ba640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 0x7f189c03b4a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f18980098c0 tx=0x7f1898006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.860+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 msgr2=0x7f18b019b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:48.456 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.860+0000 7f18b59ba640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009940 tx=0x7f18a0032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.860+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 shutdown_connections 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.861+0000 7f18b59ba640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f189c038fe0 0x7f189c03b4a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.861+0000 7f18b59ba640 1 --2- 172.21.15.129:0/1466732236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18b01074b0 0x7f18b019b350 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.861+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 >> 172.21.15.129:0/1466732236 conn(0x7f18b0102c60 msgr2=0x7f18b0103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.861+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 shutdown_connections 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:47.861+0000 7f18b59ba640 1 -- 172.21.15.129:0/1466732236 wait complete. 2024-09-05T01:05:48.457 INFO:teuthology.orchestra.run.smithi129.stdout:Enabling cephadm module... 2024-09-05T01:05:48.569 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1466732236' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T01:05:49.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2391818784' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-05T01:05:49.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ignoring --setuser ceph since I am not root 2024-09-05T01:05:49.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ignoring --setgroup ceph since I am not root 2024-09-05T01:05:49.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:49.626+0000 7f5aaca07040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:05:49.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:49.685+0000 7f5aaca07040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:05:50.067 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:49.840+0000 7f5aaca07040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:05:50.068 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:49.910+0000 7f5aaca07040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:05:50.068 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:50.066+0000 7f5aaca07040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:05:50.140 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.882+0000 7f6ad9b14640 1 Processor -- start 2024-09-05T01:05:50.140 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.882+0000 7f6ad9b14640 1 -- start start 2024-09-05T01:05:50.140 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.882+0000 7f6ad9b14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:50.140 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.882+0000 7f6ad9b14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4106d90 con 0x7f6ad41063c0 2024-09-05T01:05:50.140 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.883+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.883+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54932/0 (socket says 172.21.15.129:54932) 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.883+0000 7f6ad37fe640 1 -- 172.21.15.129:0/2760245368 learned_addr learned my addr 172.21.15.129:0/2760245368 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.883+0000 7f6ad37fe640 1 -- 172.21.15.129:0/2760245368 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41075c0 con 0x7f6ad41063c0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad37fe640 1 --2- 172.21.15.129:0/2760245368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009820 tx=0x7f6ac402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1053d08a1bf1384a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad2ffd640 1 -- 172.21.15.129:0/2760245368 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac4037070 con 0x7f6ad41063c0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad2ffd640 1 -- 172.21.15.129:0/2760245368 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6ac40328d0 con 0x7f6ad41063c0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad2ffd640 1 -- 172.21.15.129:0/2760245368 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac40307d0 con 0x7f6ad41063c0 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2760245368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 msgr2=0x7f6ad41067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2760245368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009820 tx=0x7f6ac402a270 comp rx=0 tx=0).stop 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2760245368 shutdown_connections 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2760245368 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad41067c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:50.141 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2760245368 >> 172.21.15.129:0/2760245368 conn(0x7f6ad4101b30 msgr2=0x7f6ad4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2760245368 shutdown_connections 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.884+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2760245368 wait complete. 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.885+0000 7f6ad9b14640 1 Processor -- start 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.885+0000 7f6ad9b14640 1 -- start start 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.885+0000 7f6ad9b14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.885+0000 7f6ad9b14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410a970 con 0x7f6ad41063c0 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54948/0 (socket says 172.21.15.129:54948) 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad37fe640 1 -- 172.21.15.129:0/2391818784 learned_addr learned my addr 172.21.15.129:0/2391818784 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad37fe640 1 -- 172.21.15.129:0/2391818784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad419e0b0 con 0x7f6ad41063c0 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad37fe640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009940 tx=0x7f6ac4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.886+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac4042070 con 0x7f6ad41063c0 2024-09-05T01:05:50.142 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.887+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6ac4030c80 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.887+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad419e280 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.887+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ac4037040 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.887+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad419e620 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f6ac4045070 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6ad17fa640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 0x7f6ac003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad4106840 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ac406bdf0 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6acaffd640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 0x7f6ac003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.888+0000 7f6acaffd640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 0x7f6ac003b550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0098c0 tx=0x7f6abc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:48.893+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ac4045360 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.102+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f6ad419eec0 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.479+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f6ac402f2a0 con 0x7f6ad41063c0 2024-09-05T01:05:50.143 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.480+0000 7f6ad17fa640 1 -- 172.21.15.129:0/2391818784 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f6ac403b330 con 0x7f6ad41063c0 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 msgr2=0x7f6ac003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 0x7f6ac003b550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0098c0 tx=0x7f6abc006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 msgr2=0x7f6ad4075320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009940 tx=0x7f6ac4032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 shutdown_connections 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f6ac0039090 0x7f6ac003b550 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 --2- 172.21.15.129:0/2391818784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ad41063c0 0x7f6ad4075320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 >> 172.21.15.129:0/2391818784 conn(0x7f6ad4101b30 msgr2=0x7f6ad41025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 shutdown_connections 2024-09-05T01:05:50.144 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:49.482+0000 7f6ad9b14640 1 -- 172.21.15.129:0/2391818784 wait complete. 2024-09-05T01:05:50.383 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:50.217+0000 7f5aaca07040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:05:50.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2391818784' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-05T01:05:50.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-mon[30979]: mgrmap e5: x(active, since 5s) 2024-09-05T01:05:50.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:50.545+0000 7f5aaca07040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:05:50.743 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:50.614+0000 7f5aaca07040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:05:51.112 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:50.807+0000 7f5aaca07040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:05:51.375 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.588+0000 7fcb87c1c640 1 Processor -- start 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb87c1c640 1 -- start start 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb87c1c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:51.381 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb87c1c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8007b740 con 0x7fcb8007cd10 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb859b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb859b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54988/0 (socket says 172.21.15.129:54988) 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb859b8640 1 -- 172.21.15.129:0/822965622 learned_addr learned my addr 172.21.15.129:0/822965622 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.589+0000 7fcb859b8640 1 -- 172.21.15.129:0/822965622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8007b8c0 con 0x7fcb8007cd10 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.590+0000 7fcb859b8640 1 --2- 172.21.15.129:0/822965622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009820 tx=0x7fcb7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43483dc56aa6737 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.590+0000 7fcb849b6640 1 -- 172.21.15.129:0/822965622 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c037070 con 0x7fcb8007cd10 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.590+0000 7fcb849b6640 1 -- 172.21.15.129:0/822965622 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcb7c0328d0 con 0x7fcb8007cd10 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.590+0000 7fcb849b6640 1 -- 172.21.15.129:0/822965622 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c0307d0 con 0x7fcb8007cd10 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 -- 172.21.15.129:0/822965622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 msgr2=0x7fcb8007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/822965622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009820 tx=0x7fcb7c02a270 comp rx=0 tx=0).stop 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 -- 172.21.15.129:0/822965622 shutdown_connections 2024-09-05T01:05:51.382 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/822965622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8007b170 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 -- 172.21.15.129:0/822965622 >> 172.21.15.129:0/822965622 conn(0x7fcb80102cc0 msgr2=0x7fcb801050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 -- 172.21.15.129:0/822965622 shutdown_connections 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.591+0000 7fcb87c1c640 1 -- 172.21.15.129:0/822965622 wait complete. 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb87c1c640 1 Processor -- start 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb87c1c640 1 -- start start 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb87c1c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb87c1c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8010da60 con 0x7fcb8007cd10 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb859b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb859b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55004/0 (socket says 172.21.15.129:55004) 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb859b8640 1 -- 172.21.15.129:0/803716586 learned_addr learned my addr 172.21.15.129:0/803716586 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb859b8640 1 -- 172.21.15.129:0/803716586 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8019da70 con 0x7fcb8007cd10 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.592+0000 7fcb859b8640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097f0 tx=0x7fcb7c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:51.383 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.593+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c037040 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.593+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fcb7c032bb0 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.593+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8019dd00 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.593+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb7c03a390 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.593+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb801a09f0 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fcb7c03a650 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8007b1f0 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb76ffd640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 0x7fcb6c03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb851b7640 1 -- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 msgr2=0x7fcb6c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcb7c06c3a0 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.594+0000 7fcb851b7640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 0x7fcb6c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.603+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb7c06bdc0 con 0x7fcb8007cd10 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.795+0000 7fcb851b7640 1 -- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 msgr2=0x7fcb6c03b4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:51.384 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.795+0000 7fcb851b7640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 0x7fcb6c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.822+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcb801a1070 con 0x7fcb8007cd10 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.823+0000 7fcb76ffd640 1 -- 172.21.15.129:0/803716586 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fcb7c032d60 con 0x7fcb8007cd10 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 msgr2=0x7fcb6c03b4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 0x7fcb6c03b4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 msgr2=0x7fcb8019d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c0097f0 tx=0x7fcb7c030d00 comp rx=0 tx=0).stop 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 shutdown_connections 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7fcb6c039030 0x7fcb6c03b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 --2- 172.21.15.129:0/803716586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb8007cd10 0x7fcb8019d530 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 >> 172.21.15.129:0/803716586 conn(0x7fcb80102cc0 msgr2=0x7fcb801037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 shutdown_connections 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:50.825+0000 7fcb87c1c640 1 -- 172.21.15.129:0/803716586 wait complete. 2024-09-05T01:05:51.385 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for the mgr to restart... 2024-09-05T01:05:51.386 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr epoch 5... 2024-09-05T01:05:51.649 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/803716586' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T01:05:51.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:51.721+0000 7f5aaca07040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:05:51.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:51.791+0000 7f5aaca07040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:05:52.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:52.501+0000 7f5aaca07040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:05:52.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:52.615+0000 7f5aaca07040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:05:53.035 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:52.758+0000 7f5aaca07040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:05:53.035 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:52.818+0000 7f5aaca07040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:05:53.035 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:52.967+0000 7f5aaca07040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:05:53.035 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.034+0000 7f5aaca07040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:05:53.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.093+0000 7f5aaca07040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:05:53.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.251+0000 7f5aaca07040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:05:53.316 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.315+0000 7f5aaca07040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:05:53.582 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.381+0000 7f5aaca07040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:05:53.582 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.463+0000 7f5aaca07040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:05:53.582 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.522+0000 7f5aaca07040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:05:53.582 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:53.580+0000 7f5aaca07040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:54.359+0000 7f5aaca07040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: Active manager daemon x restarted 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: Activating manager daemon x 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: osdmap e2: 0 total, 0 up, 0 in 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: mgrmap e6: x(active, starting, since 0.00313216s) 2024-09-05T01:05:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: Manager daemon x is now available 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:05:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:54 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:05:55.668 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:55 smithi129 ceph-mon[30979]: Found migration_current of "None". Setting to last migration. 2024-09-05T01:05:55.669 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:55 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:55.669 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:55 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:55.669 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:55 smithi129 ceph-mon[30979]: mgrmap e7: x(active, since 1.00572s) 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8ccf18c640 1 Processor -- start 2024-09-05T01:05:55.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8ccf18c640 1 -- start start 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8ccf18c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8ccf18c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8107e70 con 0x7f8cc81074a0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8cccf28640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8cccf28640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55006/0 (socket says 172.21.15.129:55006) 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8cccf28640 1 -- 172.21.15.129:0/3324681359 learned_addr learned my addr 172.21.15.129:0/3324681359 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.828+0000 7f8cccf28640 1 -- 172.21.15.129:0/3324681359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc81086a0 con 0x7f8cc81074a0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.829+0000 7f8cccf28640 1 --2- 172.21.15.129:0/3324681359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009820 tx=0x7f8cbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68b9378b0d4200 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.829+0000 7f8cc77fe640 1 -- 172.21.15.129:0/3324681359 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc037070 con 0x7f8cc81074a0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.829+0000 7f8cc77fe640 1 -- 172.21.15.129:0/3324681359 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8cbc0328d0 con 0x7f8cc81074a0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.829+0000 7f8cc77fe640 1 -- 172.21.15.129:0/3324681359 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc0307d0 con 0x7f8cc81074a0 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 -- 172.21.15.129:0/3324681359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 msgr2=0x7f8cc81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:55.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/3324681359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009820 tx=0x7f8cbc02a270 comp rx=0 tx=0).stop 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 -- 172.21.15.129:0/3324681359 shutdown_connections 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/3324681359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc81078a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 -- 172.21.15.129:0/3324681359 >> 172.21.15.129:0/3324681359 conn(0x7f8cc8102d10 msgr2=0x7f8cc8105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 -- 172.21.15.129:0/3324681359 shutdown_connections 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.830+0000 7f8ccf18c640 1 -- 172.21.15.129:0/3324681359 wait complete. 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8ccf18c640 1 Processor -- start 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8ccf18c640 1 -- start start 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8ccf18c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8ccf18c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc810ba50 con 0x7f8cc81074a0 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8cccf28640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8cccf28640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55020/0 (socket says 172.21.15.129:55020) 2024-09-05T01:05:55.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8cccf28640 1 -- 172.21.15.129:0/1899538249 learned_addr learned my addr 172.21.15.129:0/1899538249 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8cccf28640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc819b8c0 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.831+0000 7f8cccf28640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009940 tx=0x7f8cbc032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.832+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc042070 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.832+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8cbc030c80 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.832+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc819bb50 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.832+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc037040 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.832+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc819e840 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8cbc045070 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc5ffb640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8cb803bc50 con 0x7f8cb8039080 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8cbc06beb0 con 0x7f8cc81074a0 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc7fff640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 msgr2=0x7f8cb803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:55.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:51.833+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:52.034+0000 7f8cc7fff640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 msgr2=0x7f8cb803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:52.034+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:52.435+0000 7f8cc7fff640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 msgr2=0x7f8cb803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:52.435+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:53.236+0000 7f8cc7fff640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 msgr2=0x7f8cb803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3490891928 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:53.236+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:54.362+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f8cbc06a970 con 0x7f8cc81074a0 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:54.362+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 msgr2=0x7f8cb803b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:54.362+0000 7f8cc5ffb640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.363+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f8cbc03cac0 con 0x7f8cc81074a0 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.363+0000 7f8cc5ffb640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 0x7f8cb803eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:55.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.363+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8cbc041ec0 con 0x7f8cb803c9b0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.364+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 0x7f8cb803eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.364+0000 7f8cc7fff640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 0x7f8cb803eda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4003980 tx=0x7f8cb40092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.365+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f8cbc041ec0 con 0x7f8cb803c9b0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8cc810bbd0 con 0x7f8cb803c9b0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8cc5ffb640 1 -- 172.21.15.129:0/1899538249 <== mgr.14118 v2:172.21.15.129:6800/3942407634 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8cc810bbd0 con 0x7f8cb803c9b0 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 msgr2=0x7f8cb803eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 0x7f8cb803eda0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4003980 tx=0x7f8cb40092b0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 msgr2=0x7f8cc819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009940 tx=0x7f8cbc032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 shutdown_connections 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f8cb803c9b0 0x7f8cb803eda0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:6800/3490891928,v1:172.21.15.129:6801/3490891928] conn(0x7f8cb8039080 0x7f8cb803b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 --2- 172.21.15.129:0/1899538249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cc81074a0 0x7f8cc819b380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 >> 172.21.15.129:0/1899538249 conn(0x7f8cc8102d10 msgr2=0x7f8cc8103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.372+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 shutdown_connections 2024-09-05T01:05:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:55.373+0000 7f8ccf18c640 1 -- 172.21.15.129:0/1899538249 wait complete. 2024-09-05T01:05:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:mgr epoch 5 is available 2024-09-05T01:05:55.954 INFO:teuthology.orchestra.run.smithi129.stdout:Setting orchestrator backend to cephadm... 2024-09-05T01:05:56.188 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:56.169+0000 7f5a13d62640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:01:05:56] ENGINE Error in HTTPServer.serve 2024-09-05T01:05:56.188 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: Traceback (most recent call last): 2024-09-05T01:05:56.188 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:05:56.188 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._connections.run(self.expiration_interval) 2024-09-05T01:05:56.188 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._run(expiration_interval) 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:05:56.189 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: s = self.context.wrap_socket( 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: return self.sslsocket_class._create( 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self.do_handshake() 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:05:56.190 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._sslobj.do_handshake() 2024-09-05T01:05:56.191 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:05:56.191 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:05:56.467 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-mon[30979]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T01:05:56.467 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-mon[30979]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T01:05:56.467 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:56 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.464+0000 7fa0c6174640 1 Processor -- start 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.464+0000 7fa0c6174640 1 -- start start 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0c6174640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0c6174640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0107e90 con 0x7fa0c01074c0 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54598/0 (socket says 172.21.15.129:54598) 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0bf7fe640 1 -- 172.21.15.129:0/2973756642 learned_addr learned my addr 172.21.15.129:0/2973756642 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:57.291 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.465+0000 7fa0bf7fe640 1 -- 172.21.15.129:0/2973756642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c01086c0 con 0x7fa0c01074c0 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0bf7fe640 1 --2- 172.21.15.129:0/2973756642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e80 tx=0x7fa0b802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661464d9f18265f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0be7fc640 1 -- 172.21.15.129:0/2973756642 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8036070 con 0x7fa0c01074c0 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0be7fc640 1 -- 172.21.15.129:0/2973756642 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa0b8031890 con 0x7fa0c01074c0 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0be7fc640 1 -- 172.21.15.129:0/2973756642 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8007980 con 0x7fa0c01074c0 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0c6174640 1 -- 172.21.15.129:0/2973756642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 msgr2=0x7fa0c01078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0c6174640 1 --2- 172.21.15.129:0/2973756642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009e80 tx=0x7fa0b802a7a0 comp rx=0 tx=0).stop 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0c6174640 1 -- 172.21.15.129:0/2973756642 shutdown_connections 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0c6174640 1 --2- 172.21.15.129:0/2973756642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c01078c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.466+0000 7fa0c6174640 1 -- 172.21.15.129:0/2973756642 >> 172.21.15.129:0/2973756642 conn(0x7fa0c0102cb0 msgr2=0x7fa0c01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 -- 172.21.15.129:0/2973756642 shutdown_connections 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 -- 172.21.15.129:0/2973756642 wait complete. 2024-09-05T01:05:57.292 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 Processor -- start 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 -- start start 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0c6174640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010ba70 con 0x7fa0c01074c0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.467+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54608/0 (socket says 172.21.15.129:54608) 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bf7fe640 1 -- 172.21.15.129:0/732751732 learned_addr learned my addr 172.21.15.129:0/732751732 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bf7fe640 1 -- 172.21.15.129:0/732751732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019b920 con 0x7fa0c01074c0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bf7fe640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802acc0 tx=0x7fa0b8031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8041030 con 0x7fa0c01074c0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa0b80073f0 con 0x7fa0c01074c0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019bbb0 con 0x7fa0c01074c0 2024-09-05T01:05:57.293 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8036040 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.468+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c019e8a0 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.469+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa0b8044070 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.469+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c010bbf0 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.469+0000 7fa0bcff9640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 0x7fa0ac03b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.469+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0b8042070 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.469+0000 7fa0beffd640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 0x7fa0ac03b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.470+0000 7fa0beffd640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 0x7fa0ac03b540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098c0 tx=0x7fa0a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.474+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0b804b070 con 0x7fa0c01074c0 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.654+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fa0c0103e40 con 0x7fa0ac039080 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.660+0000 7fa0bcff9640 1 -- 172.21.15.129:0/732751732 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa0c0103e40 con 0x7fa0ac039080 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 msgr2=0x7fa0ac03b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:57.294 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 0x7fa0ac03b540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80098c0 tx=0x7fa0a8006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 msgr2=0x7fa0c019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa0b802acc0 tx=0x7fa0b8031c70 comp rx=0 tx=0).stop 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 shutdown_connections 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa0ac039080 0x7fa0ac03b540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 --2- 172.21.15.129:0/732751732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa0c01074c0 0x7fa0c019b3e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 >> 172.21.15.129:0/732751732 conn(0x7fa0c0102cb0 msgr2=0x7fa0c0103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 shutdown_connections 2024-09-05T01:05:57.295 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:56.662+0000 7fa0c6174640 1 -- 172.21.15.129:0/732751732 wait complete. 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: [05/Sep/2024:01:05:55] ENGINE Bus STARTING 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: [05/Sep/2024:01:05:56] ENGINE Serving on http://172.21.15.129:8765 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: [05/Sep/2024:01:05:56] ENGINE Serving on https://172.21.15.129:7150 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: [05/Sep/2024:01:05:56] ENGINE Bus STARTED 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: [05/Sep/2024:01:05:56] ENGINE Error in HTTPServer.serve 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout: Traceback (most recent call last): 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:05:57.539 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._connections.run(self.expiration_interval) 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._run(expiration_interval) 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: s = self.context.wrap_socket( 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: return self.sslsocket_class._create( 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: self.do_handshake() 2024-09-05T01:05:57.540 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:05:57.541 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._sslobj.do_handshake() 2024-09-05T01:05:57.541 INFO:journalctl@ceph.mon.a.smithi129.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:05:57.541 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: mgrmap e8: x(active, since 2s) 2024-09-05T01:05:57.541 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:57.541 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:57 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:05:58.473 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:58 smithi129 ceph-mon[30979]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.717+0000 7fb6bf2ec640 1 Processor -- start 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.717+0000 7fb6bf2ec640 1 -- start start 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.717+0000 7fb6bf2ec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.717+0000 7fb6bf2ec640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b807c850 con 0x7fb6b807de20 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.718+0000 7fb6bd088640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.718+0000 7fb6bd088640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54616/0 (socket says 172.21.15.129:54616) 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.718+0000 7fb6bd088640 1 -- 172.21.15.129:0/1301414827 learned_addr learned my addr 172.21.15.129:0/1301414827 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:58.481 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.718+0000 7fb6bd088640 1 -- 172.21.15.129:0/1301414827 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b807c9d0 con 0x7fb6b807de20 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6bd088640 1 --2- 172.21.15.129:0/1301414827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009820 tx=0x7fb6b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3ec50a66fb1f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6affff640 1 -- 172.21.15.129:0/1301414827 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4037070 con 0x7fb6b807de20 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6affff640 1 -- 172.21.15.129:0/1301414827 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6b40328d0 con 0x7fb6b807de20 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6affff640 1 -- 172.21.15.129:0/1301414827 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b40307d0 con 0x7fb6b807de20 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/1301414827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 msgr2=0x7fb6b807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.719+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/1301414827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009820 tx=0x7fb6b402a270 comp rx=0 tx=0).stop 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/1301414827 shutdown_connections 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/1301414827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b807c280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/1301414827 >> 172.21.15.129:0/1301414827 conn(0x7fb6b8102cc0 msgr2=0x7fb6b81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/1301414827 shutdown_connections 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/1301414827 wait complete. 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 Processor -- start 2024-09-05T01:05:58.482 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- start start 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.720+0000 7fb6bf2ec640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b810dac0 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bd088640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bd088640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54622/0 (socket says 172.21.15.129:54622) 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bd088640 1 -- 172.21.15.129:0/854510517 learned_addr learned my addr 172.21.15.129:0/854510517 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bd088640 1 -- 172.21.15.129:0/854510517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819da90 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bd088640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4032e10 tx=0x7fb6b4032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4042070 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb6b4030c80 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b819dd20 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4037040 con 0x7fb6b807de20 2024-09-05T01:05:58.483 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.721+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b81a0a10 con 0x7fb6b807de20 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.722+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb6b4045070 con 0x7fb6b807de20 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.722+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b807c300 con 0x7fb6b807de20 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.723+0000 7fb6ae7fc640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 0x7fb6a403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.723+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb6b406be90 con 0x7fb6b807de20 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.723+0000 7fb6bc887640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 0x7fb6a403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.723+0000 7fb6bc887640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 0x7fb6a403b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098c0 tx=0x7fb6a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.727+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6b4045360 con 0x7fb6b807de20 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.895+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb6b8103ea0 con 0x7fb6a4039080 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.896+0000 7fb6ae7fc640 1 -- 172.21.15.129:0/854510517 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb6b8103ea0 con 0x7fb6a4039080 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 msgr2=0x7fb6a403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 0x7fb6a403b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098c0 tx=0x7fb6a8006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 msgr2=0x7fb6b819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:58.484 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4032e10 tx=0x7fb6b4032e40 comp rx=0 tx=0).stop 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 shutdown_connections 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb6a4039080 0x7fb6a403b540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 --2- 172.21.15.129:0/854510517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb6b807de20 0x7fb6b819d550 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 >> 172.21.15.129:0/854510517 conn(0x7fb6b8102cc0 msgr2=0x7fb6b8103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 shutdown_connections 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:57.898+0000 7fb6bf2ec640 1 -- 172.21.15.129:0/854510517 wait complete. 2024-09-05T01:05:58.485 INFO:teuthology.orchestra.run.smithi129.stdout:Generating ssh key... 2024-09-05T01:05:59.422 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: Generating public/private rsa key pair. 2024-09-05T01:05:59.422 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: Your identification has been saved in /tmp/tmp1f0qi8fw/key 2024-09-05T01:05:59.422 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: Your public key has been saved in /tmp/tmp1f0qi8fw/key.pub 2024-09-05T01:05:59.422 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: The key fingerprint is: 2024-09-05T01:05:59.422 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: SHA256:yXR7sokf0MrIdX0LCVmOy3FsCzML2bvJEzuiElajKIA ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: The key's randomart image is: 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: +---[RSA 3072]----+ 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: | . | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: | o * | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: |. + @ = | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: |E oo * # o | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: |. . o .S @ * . | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: |. . +. + * X o . | 2024-09-05T01:05:59.423 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: | . . .o = @ . | 2024-09-05T01:05:59.424 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: | . . o + | 2024-09-05T01:05:59.424 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: | .. . | 2024-09-05T01:05:59.424 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: +----[SHA256]-----+ 2024-09-05T01:05:59.713 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-mon[30979]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:05:59.713 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:59.713 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:05:59 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:05:59.719 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.859+0000 7f70ce1c5640 1 Processor -- start 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.859+0000 7f70ce1c5640 1 -- start start 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70ce1c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70ce1c5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8107e80 con 0x7f70c81074b0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54636/0 (socket says 172.21.15.129:54636) 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70c77fe640 1 -- 172.21.15.129:0/233118286 learned_addr learned my addr 172.21.15.129:0/233118286 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.860+0000 7f70c77fe640 1 -- 172.21.15.129:0/233118286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81086b0 con 0x7f70c81074b0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.861+0000 7f70c77fe640 1 --2- 172.21.15.129:0/233118286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009820 tx=0x7f70b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33416adb4ad016e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.861+0000 7f70c6ffd640 1 -- 172.21.15.129:0/233118286 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b8037070 con 0x7f70c81074b0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.861+0000 7f70c6ffd640 1 -- 172.21.15.129:0/233118286 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70b80328d0 con 0x7f70c81074b0 2024-09-05T01:05:59.720 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.861+0000 7f70c6ffd640 1 -- 172.21.15.129:0/233118286 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b80307d0 con 0x7f70c81074b0 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- 172.21.15.129:0/233118286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 msgr2=0x7f70c81078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/233118286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009820 tx=0x7f70b802a270 comp rx=0 tx=0).stop 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- 172.21.15.129:0/233118286 shutdown_connections 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/233118286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c81078b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- 172.21.15.129:0/233118286 >> 172.21.15.129:0/233118286 conn(0x7f70c8102c40 msgr2=0x7f70c8105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- 172.21.15.129:0/233118286 shutdown_connections 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- 172.21.15.129:0/233118286 wait complete. 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 Processor -- start 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- start start 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70ce1c5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810ba60 con 0x7f70c81074b0 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:59.721 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.862+0000 7f70c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54644/0 (socket says 172.21.15.129:54644) 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c77fe640 1 -- 172.21.15.129:0/337037878 learned_addr learned my addr 172.21.15.129:0/337037878 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c77fe640 1 -- 172.21.15.129:0/337037878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c819b870 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c77fe640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009940 tx=0x7f70b8032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b8042070 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70b8030c80 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c819bb00 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70b8037040 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.863+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c819e7f0 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.864+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f70b8045070 con 0x7f70c81074b0 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.864+0000 7f70c57fa640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 0x7f70b403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:05:59.722 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.864+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c810bbe0 con 0x7f70c81074b0 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.864+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70b806c080 con 0x7f70c81074b0 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.864+0000 7f70bffff640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 0x7f70b403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.865+0000 7f70bffff640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 0x7f70b403b4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098c0 tx=0x7f70b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:58.869+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70b803ac60 con 0x7f70c81074b0 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.036+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f70c8103d50 con 0x7f70b4039030 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.139+0000 7f70c57fa640 1 -- 172.21.15.129:0/337037878 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f70c8103d50 con 0x7f70b4039030 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.141+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 msgr2=0x7f70b403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.141+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 0x7f70b403b4f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f70b00098c0 tx=0x7f70b0006db0 comp rx=0 tx=0).stop 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.141+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 msgr2=0x7f70c819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.141+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009940 tx=0x7f70b8032e10 comp rx=0 tx=0).stop 2024-09-05T01:05:59.723 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 shutdown_connections 2024-09-05T01:05:59.724 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f70b4039030 0x7f70b403b4f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:59.724 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 --2- 172.21.15.129:0/337037878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70c81074b0 0x7f70c819b330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:05:59.724 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 >> 172.21.15.129:0/337037878 conn(0x7f70c8102c40 msgr2=0x7f70c8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:05:59.724 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 shutdown_connections 2024-09-05T01:05:59.724 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:05:59.142+0000 7f70ce1c5640 1 -- 172.21.15.129:0/337037878 wait complete. 2024-09-05T01:06:00.676 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:00 smithi129 ceph-mon[30979]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:00.676 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:00 smithi129 ceph-mon[30979]: Generating ssh key... 2024-09-05T01:06:00.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.106+0000 7fb2ba2aa640 1 Processor -- start 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2ba2aa640 1 -- start start 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2ba2aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2ba2aa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4107f40 con 0x7fb2b4107570 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:00.950 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54652/0 (socket says 172.21.15.129:54652) 2024-09-05T01:06:00.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2b3fff640 1 -- 172.21.15.129:0/4199113563 learned_addr learned my addr 172.21.15.129:0/4199113563 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:00.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.107+0000 7fb2b3fff640 1 -- 172.21.15.129:0/4199113563 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b4108770 con 0x7fb2b4107570 2024-09-05T01:06:00.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.108+0000 7fb2b3fff640 1 --2- 172.21.15.129:0/4199113563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009820 tx=0x7fb2a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c211f4c149ebe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:00.951 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.108+0000 7fb2b2ffd640 1 -- 172.21.15.129:0/4199113563 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2a4037070 con 0x7fb2b4107570 2024-09-05T01:06:00.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.108+0000 7fb2b2ffd640 1 -- 172.21.15.129:0/4199113563 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb2a40328d0 con 0x7fb2b4107570 2024-09-05T01:06:00.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.108+0000 7fb2b2ffd640 1 -- 172.21.15.129:0/4199113563 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2a40307d0 con 0x7fb2b4107570 2024-09-05T01:06:00.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/4199113563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 msgr2=0x7fb2b4107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:00.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/4199113563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009820 tx=0x7fb2a402a270 comp rx=0 tx=0).stop 2024-09-05T01:06:00.952 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/4199113563 shutdown_connections 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/4199113563 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b4107970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/4199113563 >> 172.21.15.129:0/4199113563 conn(0x7fb2b4102da0 msgr2=0x7fb2b41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/4199113563 shutdown_connections 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/4199113563 wait complete. 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 Processor -- start 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.109+0000 7fb2ba2aa640 1 -- start start 2024-09-05T01:06:00.953 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2ba2aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2ba2aa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410bb20 con 0x7fb2b4107570 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54656/0 (socket says 172.21.15.129:54656) 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b3fff640 1 -- 172.21.15.129:0/32652488 learned_addr learned my addr 172.21.15.129:0/32652488 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b3fff640 1 -- 172.21.15.129:0/32652488 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b419b940 con 0x7fb2b4107570 2024-09-05T01:06:00.954 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b3fff640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009940 tx=0x7fb2a4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2a4042070 con 0x7fb2b4107570 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb2a4030c80 con 0x7fb2b4107570 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b419bbd0 con 0x7fb2b4107570 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2a4037040 con 0x7fb2b4107570 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.110+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b419e8c0 con 0x7fb2b4107570 2024-09-05T01:06:00.955 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.111+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb2a4045070 con 0x7fb2b4107570 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.111+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b41079f0 con 0x7fb2b4107570 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.111+0000 7fb2b17fa640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 0x7fb2a003b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.111+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb2a406bea0 con 0x7fb2b4107570 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.111+0000 7fb2b37fe640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 0x7fb2a003b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.112+0000 7fb2b37fe640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 0x7fb2a003b540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0098c0 tx=0x7fb29c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:00.956 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.116+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2a4045360 con 0x7fb2b4107570 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.303+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb2b4103ea0 con 0x7fb2a0039080 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.304+0000 7fb2b17fa640 1 -- 172.21.15.129:0/32652488 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb2b4103ea0 con 0x7fb2a0039080 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 msgr2=0x7fb2a003b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 0x7fb2a003b540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0098c0 tx=0x7fb29c006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 msgr2=0x7fb2b419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009940 tx=0x7fb2a4032e10 comp rx=0 tx=0).stop 2024-09-05T01:06:00.957 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 shutdown_connections 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb2a0039080 0x7fb2a003b540 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 --2- 172.21.15.129:0/32652488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2b4107570 0x7fb2b419b400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.306+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 >> 172.21.15.129:0/32652488 conn(0x7fb2b4102da0 msgr2=0x7fb2b4103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.307+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 shutdown_connections 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:00.307+0000 7fb2ba2aa640 1 -- 172.21.15.129:0/32652488 wait complete. 2024-09-05T01:06:00.958 INFO:teuthology.orchestra.run.smithi129.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-05T01:06:00.959 INFO:teuthology.orchestra.run.smithi129.stdout:Adding key to root@localhost authorized_keys... 2024-09-05T01:06:00.959 INFO:teuthology.orchestra.run.smithi129.stdout:Adding host smithi129... 2024-09-05T01:06:01.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:01 smithi129 ceph-mon[30979]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:02.563 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:02 smithi129 ceph-mon[30979]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "addr": "172.21.15.129", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:03.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:03 smithi129 ceph-mon[30979]: Deploying cephadm binary to smithi129 2024-09-05T01:06:05.255 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Added host 'smithi129' with addr '172.21.15.129' 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafcd30b640 1 Processor -- start 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafcd30b640 1 -- start start 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafcd30b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafcd30b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8105c60 con 0x7fafc8105320 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54664/0 (socket says 172.21.15.129:54664) 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafc6ffd640 1 -- 172.21.15.129:0/721784986 learned_addr learned my addr 172.21.15.129:0/721784986 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.319+0000 7fafc6ffd640 1 -- 172.21.15.129:0/721784986 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc8105de0 con 0x7fafc8105320 2024-09-05T01:06:05.256 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.320+0000 7fafc6ffd640 1 --2- 172.21.15.129:0/721784986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009820 tx=0x7fafb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8803053edcbbeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.320+0000 7fafc67fc640 1 -- 172.21.15.129:0/721784986 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8037070 con 0x7fafc8105320 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.320+0000 7fafc67fc640 1 -- 172.21.15.129:0/721784986 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fafb80328d0 con 0x7fafc8105320 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.320+0000 7fafc67fc640 1 -- 172.21.15.129:0/721784986 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb80307d0 con 0x7fafc8105320 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- 172.21.15.129:0/721784986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 msgr2=0x7fafc8105720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 --2- 172.21.15.129:0/721784986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009820 tx=0x7fafb802a270 comp rx=0 tx=0).stop 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- 172.21.15.129:0/721784986 shutdown_connections 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 --2- 172.21.15.129:0/721784986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc8105720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- 172.21.15.129:0/721784986 >> 172.21.15.129:0/721784986 conn(0x7fafc8100fb0 msgr2=0x7fafc81033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- 172.21.15.129:0/721784986 shutdown_connections 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- 172.21.15.129:0/721784986 wait complete. 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 Processor -- start 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 -- start start 2024-09-05T01:06:05.257 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.321+0000 7fafcd30b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafcd30b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fafc810b970 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54666/0 (socket says 172.21.15.129:54666) 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc6ffd640 1 -- 172.21.15.129:0/1267893681 learned_addr learned my addr 172.21.15.129:0/1267893681 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc6ffd640 1 -- 172.21.15.129:0/1267893681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc819b970 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc6ffd640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafb8032e10 tx=0x7fafb8032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8042070 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fafb8030c80 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc819bc00 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fafb8037040 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.322+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc819e8f0 con 0x7fafc8105320 2024-09-05T01:06:05.258 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.323+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fafb8045070 con 0x7fafc8105320 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.323+0000 7fafc4ff9640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 0x7fafb403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.323+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fafb806bf60 con 0x7fafc8105320 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.323+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc81057a0 con 0x7fafc8105320 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.323+0000 7fafbe7fc640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 0x7fafb403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.324+0000 7fafbe7fc640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 0x7fafb403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafb00098c0 tx=0x7fafb0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.328+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafb8045360 con 0x7fafc8105320 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:01.502+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "addr": "172.21.15.129", "target": ["mon-mgr", ""]}) -- 0x7fafc8102140 con 0x7fafb4039030 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.692+0000 7fafc4ff9640 1 -- 172.21.15.129:0/1267893681 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fafc8102140 con 0x7fafb4039030 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 msgr2=0x7fafb403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 0x7fafb403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafb00098c0 tx=0x7fafb0006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 msgr2=0x7fafc819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:05.259 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fafb8032e10 tx=0x7fafb8032e40 comp rx=0 tx=0).stop 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 shutdown_connections 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fafb4039030 0x7fafb403b4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 --2- 172.21.15.129:0/1267893681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fafc8105320 0x7fafc819b430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 >> 172.21.15.129:0/1267893681 conn(0x7fafc8100fb0 msgr2=0x7fafc8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 shutdown_connections 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:04.695+0000 7fafcd30b640 1 -- 172.21.15.129:0/1267893681 wait complete. 2024-09-05T01:06:05.260 INFO:teuthology.orchestra.run.smithi129.stdout:Deploying unmanaged mon service... 2024-09-05T01:06:05.901 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:05 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:05.902 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:05 smithi129 ceph-mon[30979]: Added host smithi129 2024-09-05T01:06:05.902 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:05 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb409b11640 1 Processor -- start 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb409b11640 1 -- start start 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb409b11640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb409b11640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb404107f40 con 0x7fb404107570 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb4037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb4037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54226/0 (socket says 172.21.15.129:54226) 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb4037fe640 1 -- 172.21.15.129:0/360728008 learned_addr learned my addr 172.21.15.129:0/360728008 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.713+0000 7fb4037fe640 1 -- 172.21.15.129:0/360728008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb404108770 con 0x7fb404107570 2024-09-05T01:06:06.438 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.714+0000 7fb4037fe640 1 --2- 172.21.15.129:0/360728008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009820 tx=0x7fb3f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b591dbde1fb2037 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.714+0000 7fb402ffd640 1 -- 172.21.15.129:0/360728008 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4037070 con 0x7fb404107570 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.714+0000 7fb402ffd640 1 -- 172.21.15.129:0/360728008 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3f40328d0 con 0x7fb404107570 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.714+0000 7fb402ffd640 1 -- 172.21.15.129:0/360728008 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f40307d0 con 0x7fb404107570 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- 172.21.15.129:0/360728008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 msgr2=0x7fb404107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 --2- 172.21.15.129:0/360728008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009820 tx=0x7fb3f402a270 comp rx=0 tx=0).stop 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- 172.21.15.129:0/360728008 shutdown_connections 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 --2- 172.21.15.129:0/360728008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb404107970 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- 172.21.15.129:0/360728008 >> 172.21.15.129:0/360728008 conn(0x7fb404102da0 msgr2=0x7fb4041051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- 172.21.15.129:0/360728008 shutdown_connections 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- 172.21.15.129:0/360728008 wait complete. 2024-09-05T01:06:06.439 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 Processor -- start 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.715+0000 7fb409b11640 1 -- start start 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb409b11640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb409b11640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb40410bb20 con 0x7fb404107570 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54236/0 (socket says 172.21.15.129:54236) 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4037fe640 1 -- 172.21.15.129:0/3050232669 learned_addr learned my addr 172.21.15.129:0/3050232669 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4037fe640 1 -- 172.21.15.129:0/3050232669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40419b940 con 0x7fb404107570 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4037fe640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009940 tx=0x7fb3f4032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4042070 con 0x7fb404107570 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb3f4030c80 con 0x7fb404107570 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40419bbd0 con 0x7fb404107570 2024-09-05T01:06:06.440 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3f4037040 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.716+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40419e8c0 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.717+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb3f4045070 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.717+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4041079f0 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.717+0000 7fb4017fa640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 0x7fb3f003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.717+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb3f406bfb0 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.717+0000 7fb3faffd640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 0x7fb3f003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.718+0000 7fb3faffd640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 0x7fb3f003b4f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098c0 tx=0x7fb3ec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.722+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3f4045360 con 0x7fb404107570 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.902+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb404103ea0 con 0x7fb3f0039030 2024-09-05T01:06:06.441 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.910+0000 7fb4017fa640 1 -- 172.21.15.129:0/3050232669 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb404103ea0 con 0x7fb3f0039030 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 msgr2=0x7fb3f003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 0x7fb3f003b4f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0098c0 tx=0x7fb3ec006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 msgr2=0x7fb40419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009940 tx=0x7fb3f4032e10 comp rx=0 tx=0).stop 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 shutdown_connections 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fb3f0039030 0x7fb3f003b4f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 --2- 172.21.15.129:0/3050232669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb404107570 0x7fb40419b400 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:06.442 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 >> 172.21.15.129:0/3050232669 conn(0x7fb404102da0 msgr2=0x7fb404103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:06.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 shutdown_connections 2024-09-05T01:06:06.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:05.912+0000 7fb409b11640 1 -- 172.21.15.129:0/3050232669 wait complete. 2024-09-05T01:06:06.443 INFO:teuthology.orchestra.run.smithi129.stdout:Deploying unmanaged mgr service... 2024-09-05T01:06:07.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:06 smithi129 ceph-mon[30979]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:07.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:06 smithi129 ceph-mon[30979]: Saving service mon spec with placement count:5 2024-09-05T01:06:07.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:06 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:07.942 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-05T01:06:07.942 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd9f52640 1 Processor -- start 2024-09-05T01:06:07.942 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd9f52640 1 -- start start 2024-09-05T01:06:07.942 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd9f52640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd9f52640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd4107f40 con 0x7f4bd4107570 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54246/0 (socket says 172.21.15.129:54246) 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd37fe640 1 -- 172.21.15.129:0/2520824535 learned_addr learned my addr 172.21.15.129:0/2520824535 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.063+0000 7f4bd37fe640 1 -- 172.21.15.129:0/2520824535 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd4108770 con 0x7f4bd4107570 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.064+0000 7f4bd37fe640 1 --2- 172.21.15.129:0/2520824535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009e80 tx=0x7f4bc402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca214846759e57b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.064+0000 7f4bd27fc640 1 -- 172.21.15.129:0/2520824535 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc4036070 con 0x7f4bd4107570 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd27fc640 1 -- 172.21.15.129:0/2520824535 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4bc4031890 con 0x7f4bd4107570 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd27fc640 1 -- 172.21.15.129:0/2520824535 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc4007980 con 0x7f4bd4107570 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 -- 172.21.15.129:0/2520824535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 msgr2=0x7f4bd4107970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:07.943 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/2520824535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009e80 tx=0x7f4bc402a7a0 comp rx=0 tx=0).stop 2024-09-05T01:06:07.944 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 -- 172.21.15.129:0/2520824535 shutdown_connections 2024-09-05T01:06:07.944 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/2520824535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd4107970 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:07.944 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 -- 172.21.15.129:0/2520824535 >> 172.21.15.129:0/2520824535 conn(0x7f4bd4102da0 msgr2=0x7f4bd41051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:07.944 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 -- 172.21.15.129:0/2520824535 shutdown_connections 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.065+0000 7f4bd9f52640 1 -- 172.21.15.129:0/2520824535 wait complete. 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd9f52640 1 Processor -- start 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd9f52640 1 -- start start 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd9f52640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd9f52640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd410bb20 con 0x7f4bd4107570 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:07.945 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54248/0 (socket says 172.21.15.129:54248) 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.066+0000 7f4bd37fe640 1 -- 172.21.15.129:0/928116416 learned_addr learned my addr 172.21.15.129:0/928116416 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd37fe640 1 -- 172.21.15.129:0/928116416 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd419b940 con 0x7f4bd4107570 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd37fe640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007bb0 tx=0x7f4bc4031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc4041020 con 0x7f4bd4107570 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd419bbd0 con 0x7f4bd4107570 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f4bc403c900 con 0x7f4bd4107570 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bc4036040 con 0x7f4bd4107570 2024-09-05T01:06:07.946 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.067+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd419e8c0 con 0x7f4bd4107570 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.068+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4bc4044020 con 0x7f4bd4107570 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.068+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd41079f0 con 0x7f4bd4107570 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.068+0000 7f4bd0ff9640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 0x7f4bc003b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.068+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4bc406d680 con 0x7f4bd4107570 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.068+0000 7f4bd2ffd640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 0x7f4bc003b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.069+0000 7f4bd2ffd640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 0x7f4bc003b4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098c0 tx=0x7f4bbc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.073+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4bc404b020 con 0x7f4bd4107570 2024-09-05T01:06:07.947 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.263+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f4bd4103ea0 con 0x7f4bc0039030 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.270+0000 7f4bd0ff9640 1 -- 172.21.15.129:0/928116416 <== mgr.14118 v2:172.21.15.129:6800/3942407634 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4bd4103ea0 con 0x7f4bc0039030 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.272+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 msgr2=0x7f4bc003b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 0x7f4bc003b4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc0098c0 tx=0x7f4bbc006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 msgr2=0x7f4bd419b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4007bb0 tx=0x7f4bc4031c70 comp rx=0 tx=0).stop 2024-09-05T01:06:07.948 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 shutdown_connections 2024-09-05T01:06:07.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f4bc0039030 0x7f4bc003b4f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:07.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 --2- 172.21.15.129:0/928116416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4bd4107570 0x7f4bd419b400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:07.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 >> 172.21.15.129:0/928116416 conn(0x7f4bd4102da0 msgr2=0x7f4bd4103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:07.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 shutdown_connections 2024-09-05T01:06:07.949 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:07.273+0000 7f4bd9f52640 1 -- 172.21.15.129:0/928116416 wait complete. 2024-09-05T01:06:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:08 smithi129 ceph-mon[30979]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:08 smithi129 ceph-mon[30979]: Saving service mgr spec with placement count:2 2024-09-05T01:06:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:08 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:09.520 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.371+0000 7f5bf753e640 1 Processor -- start 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf753e640 1 -- start start 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf753e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf753e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf007be10 con 0x7f5bf0108050 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf52da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf52da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54258/0 (socket says 172.21.15.129:54258) 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf52da640 1 -- 172.21.15.129:0/1621170694 learned_addr learned my addr 172.21.15.129:0/1621170694 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.372+0000 7f5bf52da640 1 -- 172.21.15.129:0/1621170694 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf010a980 con 0x7f5bf0108050 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5bf52da640 1 --2- 172.21.15.129:0/1621170694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009820 tx=0x7f5bec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145a562a1c34bcab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5be3fff640 1 -- 172.21.15.129:0/1621170694 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bec037070 con 0x7f5bf0108050 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5be3fff640 1 -- 172.21.15.129:0/1621170694 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5bec0328d0 con 0x7f5bf0108050 2024-09-05T01:06:09.521 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5be3fff640 1 -- 172.21.15.129:0/1621170694 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bec0307d0 con 0x7f5bf0108050 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5bf753e640 1 -- 172.21.15.129:0/1621170694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 msgr2=0x7f5bf010a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.373+0000 7f5bf753e640 1 --2- 172.21.15.129:0/1621170694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5bec009820 tx=0x7f5bec02a270 comp rx=0 tx=0).stop 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- 172.21.15.129:0/1621170694 shutdown_connections 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 --2- 172.21.15.129:0/1621170694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf010a440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- 172.21.15.129:0/1621170694 >> 172.21.15.129:0/1621170694 conn(0x7f5bf0101c90 msgr2=0x7f5bf01040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- 172.21.15.129:0/1621170694 shutdown_connections 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- 172.21.15.129:0/1621170694 wait complete. 2024-09-05T01:06:09.522 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 Processor -- start 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- start start 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.374+0000 7f5bf753e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf010b960 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf52da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf52da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54260/0 (socket says 172.21.15.129:54260) 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf52da640 1 -- 172.21.15.129:0/694181393 learned_addr learned my addr 172.21.15.129:0/694181393 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf52da640 1 -- 172.21.15.129:0/694181393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf019b950 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf52da640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5bec032e10 tx=0x7f5bec032e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bec042070 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5bec030c80 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf019bbe0 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bec037040 con 0x7f5bf0108050 2024-09-05T01:06:09.523 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.375+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf019e8d0 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.376+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5bec045070 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.376+0000 7f5be27fc640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 0x7f5bdc03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.376+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5bec06bff0 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.376+0000 7f5bf4ad9640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 0x7f5bdc03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.376+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf01084b0 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.377+0000 7f5bf4ad9640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 0x7f5bdc03b4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098c0 tx=0x7f5be4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.381+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bec045360 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.549+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f5bf019eaa0 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.554+0000 7f5be27fc640 1 -- 172.21.15.129:0/694181393 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f5bec06b980 con 0x7f5bf0108050 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 msgr2=0x7f5bdc03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 0x7f5bdc03b4f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5be40098c0 tx=0x7f5be4006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:09.524 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 msgr2=0x7f5bf019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5bec032e10 tx=0x7f5bec032e40 comp rx=0 tx=0).stop 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 shutdown_connections 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f5bdc039030 0x7f5bdc03b4f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 --2- 172.21.15.129:0/694181393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5bf0108050 0x7f5bf019b410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.556+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 >> 172.21.15.129:0/694181393 conn(0x7f5bf0101c90 msgr2=0x7f5bf01027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.557+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 shutdown_connections 2024-09-05T01:06:09.525 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:08.557+0000 7f5bf753e640 1 -- 172.21.15.129:0/694181393 wait complete. 2024-09-05T01:06:09.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:09 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/694181393' entity='client.admin' 2024-09-05T01:06:09.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:09 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:09.767 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:09 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:11.290 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:11 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1600645646' entity='client.admin' 2024-09-05T01:06:11.290 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:11 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f07a12f5640 1 Processor -- start 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f07a12f5640 1 -- start start 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f07a12f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f07a12f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f079c106d80 con 0x7f079c1063b0 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f079affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f079affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54274/0 (socket says 172.21.15.129:54274) 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.954+0000 7f079affd640 1 -- 172.21.15.129:0/2605016188 learned_addr learned my addr 172.21.15.129:0/2605016188 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.955+0000 7f079affd640 1 -- 172.21.15.129:0/2605016188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c1075b0 con 0x7f079c1063b0 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.955+0000 7f079affd640 1 --2- 172.21.15.129:0/2605016188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f078c009820 tx=0x7f078c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaa2cf0d43424f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:11.300 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.955+0000 7f079a7fc640 1 -- 172.21.15.129:0/2605016188 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c037070 con 0x7f079c1063b0 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.955+0000 7f079a7fc640 1 -- 172.21.15.129:0/2605016188 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f078c0328f0 con 0x7f079c1063b0 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.955+0000 7f079a7fc640 1 -- 172.21.15.129:0/2605016188 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c030740 con 0x7f079c1063b0 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 -- 172.21.15.129:0/2605016188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 msgr2=0x7f079c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 --2- 172.21.15.129:0/2605016188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f078c009820 tx=0x7f078c02a270 comp rx=0 tx=0).stop 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 -- 172.21.15.129:0/2605016188 shutdown_connections 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 --2- 172.21.15.129:0/2605016188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c1067b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 -- 172.21.15.129:0/2605016188 >> 172.21.15.129:0/2605016188 conn(0x7f079c101b60 msgr2=0x7f079c103f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 -- 172.21.15.129:0/2605016188 shutdown_connections 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.956+0000 7f07a12f5640 1 -- 172.21.15.129:0/2605016188 wait complete. 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f07a12f5640 1 Processor -- start 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f07a12f5640 1 -- start start 2024-09-05T01:06:11.301 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f07a12f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f07a12f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f079c10a960 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f079affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f079affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54288/0 (socket says 172.21.15.129:54288) 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f079affd640 1 -- 172.21.15.129:0/1600645646 learned_addr learned my addr 172.21.15.129:0/1600645646 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f079affd640 1 -- 172.21.15.129:0/1600645646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f079c19e0e0 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.957+0000 7f079affd640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f078c009940 tx=0x7f078c030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c042070 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f078c030e30 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c037040 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f079c19e2b0 con 0x7f079c1063b0 2024-09-05T01:06:11.302 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f079c19e650 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.958+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f078c045070 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.959+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079c106830 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.959+0000 7f0798ff9640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 0x7f078803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.959+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f078c06c030 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.959+0000 7f07927fc640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 0x7f078803b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.959+0000 7f07927fc640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 0x7f078803b540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f07840098c0 tx=0x7f0784006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:09.964+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f078c045360 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.143+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f079c19e8d0 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.145+0000 7f0798ff9640 1 -- 172.21.15.129:0/1600645646 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f078c02ed90 con 0x7f079c1063b0 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 msgr2=0x7f078803b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 0x7f078803b540 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f07840098c0 tx=0x7f0784006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 msgr2=0x7f079c07b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:11.303 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f078c009940 tx=0x7f078c030cc0 comp rx=0 tx=0).stop 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 shutdown_connections 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f0788039080 0x7f078803b540 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 --2- 172.21.15.129:0/1600645646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f079c1063b0 0x7f079c07b9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 >> 172.21.15.129:0/1600645646 conn(0x7f079c101b60 msgr2=0x7f079c1025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 shutdown_connections 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:10.148+0000 7f07a12f5640 1 -- 172.21.15.129:0/1600645646 wait complete. 2024-09-05T01:06:11.304 INFO:teuthology.orchestra.run.smithi129.stdout:Enabling the dashboard module... 2024-09-05T01:06:13.082 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-mon[30979]: from='mgr.14118 172.21.15.129:0/2727134941' entity='mgr.x' 2024-09-05T01:06:13.082 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1058173952' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-05T01:06:13.082 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ignoring --setuser ceph since I am not root 2024-09-05T01:06:13.082 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ignoring --setgroup ceph since I am not root 2024-09-05T01:06:13.082 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:12.931+0000 7f2793484040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:06:13.083 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:12 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:12.989+0000 7f2793484040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:06:13.353 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.139+0000 7f2793484040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:06:13.353 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.205+0000 7f2793484040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:06:13.354 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.352+0000 7f2793484040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:06:13.742 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.499+0000 7f2793484040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:06:13.760 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.740+0000 7fa27c97e640 1 Processor -- start 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27c97e640 1 -- start start 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27c97e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27c97e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa27807be10 con 0x7fa278108050 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54290/0 (socket says 172.21.15.129:54290) 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.741+0000 7fa27659c640 1 -- 172.21.15.129:0/3324063084 learned_addr learned my addr 172.21.15.129:0/3324063084 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.742+0000 7fa27659c640 1 -- 172.21.15.129:0/3324063084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27810a980 con 0x7fa278108050 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.742+0000 7fa27659c640 1 --2- 172.21.15.129:0/3324063084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e80 tx=0x7fa27002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341809865bd34d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.742+0000 7fa27559a640 1 -- 172.21.15.129:0/3324063084 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270036070 con 0x7fa278108050 2024-09-05T01:06:13.761 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.742+0000 7fa27559a640 1 -- 172.21.15.129:0/3324063084 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2700318b0 con 0x7fa278108050 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.742+0000 7fa27559a640 1 -- 172.21.15.129:0/3324063084 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270007930 con 0x7fa278108050 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 -- 172.21.15.129:0/3324063084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 msgr2=0x7fa27810a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 --2- 172.21.15.129:0/3324063084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa270009e80 tx=0x7fa27002a7a0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 -- 172.21.15.129:0/3324063084 shutdown_connections 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 --2- 172.21.15.129:0/3324063084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27810a440 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 -- 172.21.15.129:0/3324063084 >> 172.21.15.129:0/3324063084 conn(0x7fa278101c90 msgr2=0x7fa2781040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 -- 172.21.15.129:0/3324063084 shutdown_connections 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.743+0000 7fa27c97e640 1 -- 172.21.15.129:0/3324063084 wait complete. 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27c97e640 1 Processor -- start 2024-09-05T01:06:13.762 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27c97e640 1 -- start start 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27c97e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27c97e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa27810b9a0 con 0x7fa278108050 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54296/0 (socket says 172.21.15.129:54296) 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.744+0000 7fa27659c640 1 -- 172.21.15.129:0/1058173952 learned_addr learned my addr 172.21.15.129:0/1058173952 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa27659c640 1 -- 172.21.15.129:0/1058173952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27819b970 con 0x7fa278108050 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa27659c640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa270031c90 tx=0x7fa270031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa270036050 con 0x7fa278108050 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa27819bc00 con 0x7fa278108050 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa270042070 con 0x7fa278108050 2024-09-05T01:06:13.763 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa27003d9c0 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.745+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa27819e8f0 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.746+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa27003dc70 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.746+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2781084b0 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.747+0000 7fa26f7fe640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 0x7fa26403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.747+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa27006d4e0 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.747+0000 7fa275d9b640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 0x7fa26403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.747+0000 7fa275d9b640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 0x7fa26403b540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098c0 tx=0x7fa260006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.751+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa270044310 con 0x7fa278108050 2024-09-05T01:06:13.764 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:11.968+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa27819eac0 con 0x7fa278108050 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.785+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa27003bd50 con 0x7fa278108050 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.787+0000 7fa26f7fe640 1 -- 172.21.15.129:0/1058173952 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa27004dcf0 con 0x7fa278108050 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 msgr2=0x7fa26403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 0x7fa26403b540 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa2600098c0 tx=0x7fa260006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 msgr2=0x7fa27819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa270031c90 tx=0x7fa270031cc0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 shutdown_connections 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7fa264039080 0x7fa26403b540 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 --2- 172.21.15.129:0/1058173952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa278108050 0x7fa27819b430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.788+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 >> 172.21.15.129:0/1058173952 conn(0x7fa278101c90 msgr2=0x7fa2781027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:13.765 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.789+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 shutdown_connections 2024-09-05T01:06:13.766 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:12.789+0000 7fa27c97e640 1 -- 172.21.15.129:0/1058173952 wait complete. 2024-09-05T01:06:14.015 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1058173952' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-05T01:06:14.015 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-mon[30979]: mgrmap e9: x(active, since 18s) 2024-09-05T01:06:14.016 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.820+0000 7f2793484040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:06:14.016 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:13 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:13.883+0000 7f2793484040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:06:14.397 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:14 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:14.076+0000 7f2793484040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:06:14.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:14 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3364659557' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T01:06:15.000 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:06:15.000 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-05T01:06:15.000 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T01:06:15.000 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.190+0000 7ff493835640 1 Processor -- start 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.190+0000 7ff493835640 1 -- start start 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.190+0000 7ff493835640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.190+0000 7ff493835640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c075700 con 0x7ff48c107fb0 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.191+0000 7ff4915d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.191+0000 7ff4915d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53288/0 (socket says 172.21.15.129:53288) 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.191+0000 7ff4915d1640 1 -- 172.21.15.129:0/4066378659 learned_addr learned my addr 172.21.15.129:0/4066378659 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.191+0000 7ff4915d1640 1 -- 172.21.15.129:0/4066378659 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c10a8e0 con 0x7ff48c107fb0 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.191+0000 7ff4915d1640 1 --2- 172.21.15.129:0/4066378659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff488009820 tx=0x7ff48802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89601873e24d07b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:15.001 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff483fff640 1 -- 172.21.15.129:0/4066378659 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488037070 con 0x7ff48c107fb0 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff483fff640 1 -- 172.21.15.129:0/4066378659 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4880328f0 con 0x7ff48c107fb0 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff483fff640 1 -- 172.21.15.129:0/4066378659 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488030740 con 0x7ff48c107fb0 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 -- 172.21.15.129:0/4066378659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 msgr2=0x7ff48c10a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 --2- 172.21.15.129:0/4066378659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff488009820 tx=0x7ff48802a270 comp rx=0 tx=0).stop 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 -- 172.21.15.129:0/4066378659 shutdown_connections 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 --2- 172.21.15.129:0/4066378659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c10a3a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 -- 172.21.15.129:0/4066378659 >> 172.21.15.129:0/4066378659 conn(0x7ff48c101b30 msgr2=0x7ff48c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 -- 172.21.15.129:0/4066378659 shutdown_connections 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.192+0000 7ff493835640 1 -- 172.21.15.129:0/4066378659 wait complete. 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff493835640 1 Processor -- start 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff493835640 1 -- start start 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff493835640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:15.002 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff493835640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c10b8a0 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff4915d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff4915d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53296/0 (socket says 172.21.15.129:53296) 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.193+0000 7ff4915d1640 1 -- 172.21.15.129:0/3364659557 learned_addr learned my addr 172.21.15.129:0/3364659557 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff4915d1640 1 -- 172.21.15.129:0/3364659557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c107d70 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff4915d1640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff488030cc0 tx=0x7ff488030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488042070 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff488030e60 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c105f60 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff488037040 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.194+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1064a0 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7ff488045070 con 0x7ff48c107fb0 2024-09-05T01:06:15.003 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff4827fc640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 0x7ff47803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c108410 con 0x7ff48c107fb0 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff490dd0640 1 -- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 msgr2=0x7ff47803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff48806c0d0 con 0x7ff48c107fb0 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.195+0000 7ff490dd0640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 0x7ff47803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.200+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff488034e00 con 0x7ff48c107fb0 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.396+0000 7ff490dd0640 1 -- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 msgr2=0x7ff47803b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.396+0000 7ff490dd0640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 0x7ff47803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.421+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7ff48c106720 con 0x7ff48c107fb0 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.421+0000 7ff4827fc640 1 -- 172.21.15.129:0/3364659557 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7ff48802ed90 con 0x7ff48c107fb0 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 msgr2=0x7ff47803b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 0x7ff47803b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:15.004 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 msgr2=0x7ff48c107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff488030cc0 tx=0x7ff488030cf0 comp rx=0 tx=0).stop 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 shutdown_connections 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7ff478039080 0x7ff47803b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 --2- 172.21.15.129:0/3364659557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff48c107fb0 0x7ff48c107830 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 >> 172.21.15.129:0/3364659557 conn(0x7ff48c101b30 msgr2=0x7ff48c102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 shutdown_connections 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:14.423+0000 7ff493835640 1 -- 172.21.15.129:0/3364659557 wait complete. 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for the mgr to restart... 2024-09-05T01:06:15.005 INFO:teuthology.orchestra.run.smithi129.stdout:Waiting for mgr epoch 9... 2024-09-05T01:06:15.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:14 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:14.986+0000 7f2793484040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:06:15.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:15 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:15.059+0000 7f2793484040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:06:15.901 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:15 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:15.785+0000 7f2793484040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:06:16.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:15 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:15.900+0000 7f2793484040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:06:16.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.044+0000 7f2793484040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:06:16.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.105+0000 7f2793484040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:06:16.539 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.254+0000 7f2793484040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:06:16.539 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.322+0000 7f2793484040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:06:16.540 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.382+0000 7f2793484040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:06:16.540 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.538+0000 7f2793484040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:06:16.813 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.604+0000 7f2793484040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:06:16.813 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.671+0000 7f2793484040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:06:16.814 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.753+0000 7f2793484040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:06:16.814 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.812+0000 7f2793484040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:06:17.242 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:16 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:16.872+0000 7f2793484040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:06:17.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:17.662+0000 7f2793484040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:06:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: Active manager daemon x restarted 2024-09-05T01:06:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: Activating manager daemon x 2024-09-05T01:06:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: osdmap e3: 0 total, 0 up, 0 in 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: mgrmap e10: x(active, starting, since 0.0035688s) 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: Manager daemon x is now available 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T01:06:17.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T01:06:19.243 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:19.035+0000 7f26f8f4e640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:01:06:19] ENGINE Error in HTTPServer.serve 2024-09-05T01:06:19.243 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: Traceback (most recent call last): 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._connections.run(self.expiration_interval) 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._run(expiration_interval) 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: s = self.context.wrap_socket( 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: return self.sslsocket_class._create( 2024-09-05T01:06:19.244 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:06:19.245 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self.do_handshake() 2024-09-05T01:06:19.245 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:06:19.245 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: self._sslobj.do_handshake() 2024-09-05T01:06:19.245 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:06:19.245 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout { 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout } 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.421+0000 7f2402191640 1 Processor -- start 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.421+0000 7f2402191640 1 -- start start 2024-09-05T01:06:19.443 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f2402191640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f2402191640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc107ea0 con 0x7f23fc1074d0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53308/0 (socket says 172.21.15.129:53308) 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f23fb7fe640 1 -- 172.21.15.129:0/1639427585 learned_addr learned my addr 172.21.15.129:0/1639427585 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.422+0000 7f23fb7fe640 1 -- 172.21.15.129:0/1639427585 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc1086d0 con 0x7f23fc1074d0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.423+0000 7f23fb7fe640 1 --2- 172.21.15.129:0/1639427585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009820 tx=0x7f23f002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ddd3013d1a882f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.423+0000 7f23fa7fc640 1 -- 172.21.15.129:0/1639427585 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f0037070 con 0x7f23fc1074d0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.423+0000 7f23fa7fc640 1 -- 172.21.15.129:0/1639427585 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f00328f0 con 0x7f23fc1074d0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.423+0000 7f23fa7fc640 1 -- 172.21.15.129:0/1639427585 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f0030740 con 0x7f23fc1074d0 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 -- 172.21.15.129:0/1639427585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 msgr2=0x7f23fc1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:19.444 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 --2- 172.21.15.129:0/1639427585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009820 tx=0x7f23f002a270 comp rx=0 tx=0).stop 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 -- 172.21.15.129:0/1639427585 shutdown_connections 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 --2- 172.21.15.129:0/1639427585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc1078d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 -- 172.21.15.129:0/1639427585 >> 172.21.15.129:0/1639427585 conn(0x7f23fc102c80 msgr2=0x7f23fc1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 -- 172.21.15.129:0/1639427585 shutdown_connections 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.424+0000 7f2402191640 1 -- 172.21.15.129:0/1639427585 wait complete. 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f2402191640 1 Processor -- start 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f2402191640 1 -- start start 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f2402191640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f2402191640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10ba80 con 0x7f23fc1074d0 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53324/0 (socket says 172.21.15.129:53324) 2024-09-05T01:06:19.445 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.425+0000 7f23fb7fe640 1 -- 172.21.15.129:0/2197513581 learned_addr learned my addr 172.21.15.129:0/2197513581 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f23fb7fe640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc07e050 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f23fb7fe640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009940 tx=0x7f23f0030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f0042070 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc07c240 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f0030e30 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23f0037040 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.426+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc07c780 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f23f0045070 con 0x7f23fc1074d0 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f23f8ff9640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f23fc107950 con 0x7f23e4039080 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f23faffd640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 msgr2=0x7f23e403b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:19.446 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f23f006cfd0 con 0x7f23fc1074d0 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.427+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.628+0000 7f23faffd640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 msgr2=0x7f23e403b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:15.628+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:16.029+0000 7f23faffd640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 msgr2=0x7f23e403b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:16.029+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:16.830+0000 7f23faffd640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 msgr2=0x7f23e403b540 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.129:6800/3942407634 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:16.830+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:17.665+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f23f006ba80 con 0x7f23fc1074d0 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:17.665+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 msgr2=0x7f23e403b540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:17.665+0000 7f23f8ff9640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.666+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f23f003b330 con 0x7f23fc1074d0 2024-09-05T01:06:19.447 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.667+0000 7f23f8ff9640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 0x7f23e403ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.667+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f23fc107950 con 0x7f23e403c950 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.667+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 0x7f23e403ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.668+0000 7f23faffd640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 0x7f23e403ed40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f23ec003980 tx=0x7f23ec006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.669+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f23fc107950 con 0x7f23e403c950 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f23fc107950 con 0x7f23e403c950 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f23f8ff9640 1 -- 172.21.15.129:0/2197513581 <== mgr.14150 v2:172.21.15.129:6800/655841650 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f23fc107950 con 0x7f23e403c950 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 msgr2=0x7f23e403ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f2402191640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 0x7f23e403ed40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f23ec003980 tx=0x7f23ec006e10 comp rx=0 tx=0).stop 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 msgr2=0x7f23fc07db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.681+0000 7f2402191640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009940 tx=0x7f23f0030cc0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.448 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 shutdown_connections 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e403c950 0x7f23e403ed40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:6800/3942407634,v1:172.21.15.129:6801/3942407634] conn(0x7f23e4039080 0x7f23e403b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 --2- 172.21.15.129:0/2197513581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc1074d0 0x7f23fc07db10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 >> 172.21.15.129:0/2197513581 conn(0x7f23fc102c80 msgr2=0x7f23fc1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 shutdown_connections 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:18.682+0000 7f2402191640 1 -- 172.21.15.129:0/2197513581 wait complete. 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:mgr epoch 9 is available 2024-09-05T01:06:19.449 INFO:teuthology.orchestra.run.smithi129.stdout:Generating a dashboard self-signed certificate... 2024-09-05T01:06:19.691 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-mon[30979]: mgrmap e11: x(active, since 1.00587s) 2024-09-05T01:06:19.691 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:19 smithi129 ceph-mon[30979]: [05/Sep/2024:01:06:18] ENGINE Bus STARTING 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17cafb640 1 Processor -- start 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17cafb640 1 -- start start 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17cafb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17cafb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff178106d90 con 0x7ff1781063c0 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53386/0 (socket says 172.21.15.129:53386) 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.865+0000 7ff17659c640 1 -- 172.21.15.129:0/1605374944 learned_addr learned my addr 172.21.15.129:0/1605374944 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:20.700 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.866+0000 7ff17659c640 1 -- 172.21.15.129:0/1605374944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1781075c0 con 0x7ff1781063c0 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.866+0000 7ff17659c640 1 --2- 172.21.15.129:0/1605374944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff168009820 tx=0x7ff16802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da26b3b4dd42a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.866+0000 7ff175d9b640 1 -- 172.21.15.129:0/1605374944 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff168037070 con 0x7ff1781063c0 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.866+0000 7ff175d9b640 1 -- 172.21.15.129:0/1605374944 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1680328f0 con 0x7ff1781063c0 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.866+0000 7ff175d9b640 1 -- 172.21.15.129:0/1605374944 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff168030740 con 0x7ff1781063c0 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 -- 172.21.15.129:0/1605374944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 msgr2=0x7ff1781067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 --2- 172.21.15.129:0/1605374944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff168009820 tx=0x7ff16802a270 comp rx=0 tx=0).stop 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 -- 172.21.15.129:0/1605374944 shutdown_connections 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 --2- 172.21.15.129:0/1605374944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781067c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 -- 172.21.15.129:0/1605374944 >> 172.21.15.129:0/1605374944 conn(0x7ff178101b30 msgr2=0x7ff178103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 -- 172.21.15.129:0/1605374944 shutdown_connections 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.867+0000 7ff17cafb640 1 -- 172.21.15.129:0/1605374944 wait complete. 2024-09-05T01:06:20.701 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17cafb640 1 Processor -- start 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17cafb640 1 -- start start 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17cafb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17cafb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff178074a20 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53390/0 (socket says 172.21.15.129:53390) 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17659c640 1 -- 172.21.15.129:0/2633843413 learned_addr learned my addr 172.21.15.129:0/2633843413 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.868+0000 7ff17659c640 1 -- 172.21.15.129:0/2633843413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff178199700 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff17659c640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff168009940 tx=0x7ff168030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff168042070 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff168030e30 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff168037040 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff178199990 con 0x7ff1781063c0 2024-09-05T01:06:20.702 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.869+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17819c680 con 0x7ff1781063c0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff168045070 con 0x7ff1781063c0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff16f7fe640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 0x7ff16403d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff16806c1a0 con 0x7ff1781063c0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff16ffff640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 0x7ff16403d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1781067c0 con 0x7ff1781063c0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.870+0000 7ff16ffff640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 0x7ff16403d7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1600098c0 tx=0x7ff160006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:19.875+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff168045360 con 0x7ff1781063c0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.073+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff178062630 con 0x7ff16403b2e0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.129+0000 7ff16f7fe640 1 -- 172.21.15.129:0/2633843413 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff178062630 con 0x7ff16403b2e0 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 msgr2=0x7ff16403d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:20.703 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 0x7ff16403d7a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1600098c0 tx=0x7ff160006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 msgr2=0x7ff1781991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff168009940 tx=0x7ff168030cc0 comp rx=0 tx=0).stop 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 shutdown_connections 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff16403b2e0 0x7ff16403d7a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 --2- 172.21.15.129:0/2633843413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1781063c0 0x7ff1781991c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 >> 172.21.15.129:0/2633843413 conn(0x7ff178101b30 msgr2=0x7ff17818f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 shutdown_connections 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:20.131+0000 7ff17cafb640 1 -- 172.21.15.129:0/2633843413 wait complete. 2024-09-05T01:06:20.704 INFO:teuthology.orchestra.run.smithi129.stdout:Creating initial admin user... 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: [05/Sep/2024:01:06:18] ENGINE Serving on http://172.21.15.129:8765 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: [05/Sep/2024:01:06:19] ENGINE Serving on https://172.21.15.129:7150 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: [05/Sep/2024:01:06:19] ENGINE Bus STARTED 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: [05/Sep/2024:01:06:19] ENGINE Error in HTTPServer.serve 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: Traceback (most recent call last): 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._connections.run(self.expiration_interval) 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._run(expiration_interval) 2024-09-05T01:06:20.892 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: s = self.context.wrap_socket( 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: return self.sslsocket_class._create( 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: self.do_handshake() 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: self._sslobj.do_handshake() 2024-09-05T01:06:20.893 INFO:journalctl@ceph.mon.a.smithi129.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T01:06:20.894 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: mgrmap e12: x(active, since 2s) 2024-09-05T01:06:20.894 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:20.894 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:20.894 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:21.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:21 smithi129 ceph-mon[30979]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:21.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:21.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$cR.KopLZ1LPt17I.8BlLtefR2wwPbFp7Iru1k9lcm2/B1OjCTXeEi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725498381, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.077+0000 7f4eead06640 1 Processor -- start 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.077+0000 7f4eead06640 1 -- start start 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.077+0000 7f4eead06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.077+0000 7f4eead06640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4106d90 con 0x7f4ee41063c0 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.078+0000 7f4ee8aa2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:22.080 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.078+0000 7f4ee8aa2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53404/0 (socket says 172.21.15.129:53404) 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.078+0000 7f4ee8aa2640 1 -- 172.21.15.129:0/2705446978 learned_addr learned my addr 172.21.15.129:0/2705446978 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.078+0000 7f4ee8aa2640 1 -- 172.21.15.129:0/2705446978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41075c0 con 0x7f4ee41063c0 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4ee8aa2640 1 --2- 172.21.15.129:0/2705446978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009820 tx=0x7f4ed802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eebe20b05214c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4ee37fe640 1 -- 172.21.15.129:0/2705446978 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8037070 con 0x7f4ee41063c0 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4ee37fe640 1 -- 172.21.15.129:0/2705446978 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed80328f0 con 0x7f4ee41063c0 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4ee37fe640 1 -- 172.21.15.129:0/2705446978 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8030740 con 0x7f4ee41063c0 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4eead06640 1 -- 172.21.15.129:0/2705446978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 msgr2=0x7f4ee41067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.079+0000 7f4eead06640 1 --2- 172.21.15.129:0/2705446978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009820 tx=0x7f4ed802a270 comp rx=0 tx=0).stop 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- 172.21.15.129:0/2705446978 shutdown_connections 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 --2- 172.21.15.129:0/2705446978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee41067c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- 172.21.15.129:0/2705446978 >> 172.21.15.129:0/2705446978 conn(0x7f4ee4101b30 msgr2=0x7f4ee4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- 172.21.15.129:0/2705446978 shutdown_connections 2024-09-05T01:06:22.081 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- 172.21.15.129:0/2705446978 wait complete. 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 Processor -- start 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- start start 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.080+0000 7f4eead06640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee410a970 con 0x7f4ee41063c0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee8aa2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee8aa2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53406/0 (socket says 172.21.15.129:53406) 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee8aa2640 1 -- 172.21.15.129:0/2585638423 learned_addr learned my addr 172.21.15.129:0/2585638423 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee8aa2640 1 -- 172.21.15.129:0/2585638423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee419b8f0 con 0x7f4ee41063c0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee8aa2640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097f0 tx=0x7f4ed8030c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8042070 con 0x7f4ee41063c0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee419bb80 con 0x7f4ee41063c0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed8030d80 con 0x7f4ee41063c0 2024-09-05T01:06:22.082 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ed8037040 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.081+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee419e870 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.082+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4ed8045070 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.082+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee410aaf0 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.083+0000 7f4ee1ffb640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 0x7f4ed403b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.083+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4ed806c310 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.083+0000 7f4ee3fff640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 0x7f4ed403b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.083+0000 7f4ee3fff640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 0x7f4ed403b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00098c0 tx=0x7f4ed0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.088+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ed803add0 con 0x7f4ee41063c0 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.279+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4ee402d9f0 con 0x7f4ed4039080 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.507+0000 7f4ee1ffb640 1 -- 172.21.15.129:0/2585638423 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4ee402d9f0 con 0x7f4ed4039080 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 msgr2=0x7f4ed403b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:22.083 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 0x7f4ed403b540 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ed00098c0 tx=0x7f4ed0006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 msgr2=0x7f4ee419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80097f0 tx=0x7f4ed8030c70 comp rx=0 tx=0).stop 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 shutdown_connections 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ed4039080 0x7f4ed403b540 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 --2- 172.21.15.129:0/2585638423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee41063c0 0x7f4ee419b3b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 >> 172.21.15.129:0/2585638423 conn(0x7f4ee4101b30 msgr2=0x7f4ee41029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 shutdown_connections 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:21.510+0000 7f4eead06640 1 -- 172.21.15.129:0/2585638423 wait complete. 2024-09-05T01:06:22.084 INFO:teuthology.orchestra.run.smithi129.stdout:Fetching dashboard port number... 2024-09-05T01:06:22.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:22 smithi129 ceph-mon[30979]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:22.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:22 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2168543789' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-05T01:06:23.960 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stdout 8443 2024-09-05T01:06:23.960 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.478+0000 7f4f143c2640 1 Processor -- start 2024-09-05T01:06:23.960 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.478+0000 7f4f143c2640 1 -- start start 2024-09-05T01:06:23.960 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.478+0000 7f4f143c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:23.960 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.478+0000 7f4f143c2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c108430 con 0x7f4f0c107a60 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.479+0000 7f4f1215e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.479+0000 7f4f1215e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39416/0 (socket says 172.21.15.129:39416) 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.479+0000 7f4f1215e640 1 -- 172.21.15.129:0/1421147667 learned_addr learned my addr 172.21.15.129:0/1421147667 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.479+0000 7f4f1215e640 1 -- 172.21.15.129:0/1421147667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c1066c0 con 0x7f4f0c107a60 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f1215e640 1 --2- 172.21.15.129:0/1421147667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009820 tx=0x7f4f0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9b0997daba4f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f1115c640 1 -- 172.21.15.129:0/1421147667 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f08037070 con 0x7f4f0c107a60 2024-09-05T01:06:23.961 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f1115c640 1 -- 172.21.15.129:0/1421147667 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f080328f0 con 0x7f4f0c107a60 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f1115c640 1 -- 172.21.15.129:0/1421147667 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f08030740 con 0x7f4f0c107a60 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f143c2640 1 -- 172.21.15.129:0/1421147667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 msgr2=0x7f4f0c107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.480+0000 7f4f143c2640 1 --2- 172.21.15.129:0/1421147667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009820 tx=0x7f4f0802a270 comp rx=0 tx=0).stop 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- 172.21.15.129:0/1421147667 shutdown_connections 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 --2- 172.21.15.129:0/1421147667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c107e60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- 172.21.15.129:0/1421147667 >> 172.21.15.129:0/1421147667 conn(0x7f4f0c101420 msgr2=0x7f4f0c103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- 172.21.15.129:0/1421147667 shutdown_connections 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- 172.21.15.129:0/1421147667 wait complete. 2024-09-05T01:06:23.962 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 Processor -- start 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- start start 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.481+0000 7f4f143c2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c10aa80 con 0x7f4f0c107a60 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f1215e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f1215e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39426/0 (socket says 172.21.15.129:39426) 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f1215e640 1 -- 172.21.15.129:0/2168543789 learned_addr learned my addr 172.21.15.129:0/2168543789 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f1215e640 1 -- 172.21.15.129:0/2168543789 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c19e1d0 con 0x7f4f0c107a60 2024-09-05T01:06:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f1215e640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009940 tx=0x7f4f08030cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f08042070 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f08030e30 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c19e3a0 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f08037040 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.482+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c19e740 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.483+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4f08045070 con 0x7f4f0c107a60 2024-09-05T01:06:23.964 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.483+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c107ee0 con 0x7f4f0c107a60 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.483+0000 7f4f037fe640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 0x7f4ef803b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.483+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4f0806c130 con 0x7f4f0c107a60 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.483+0000 7f4f1195d640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 0x7f4ef803b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.484+0000 7f4f1195d640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 0x7f4ef803b540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098c0 tx=0x7f4efc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.488+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f0804b170 con 0x7f4f0c107a60 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.658+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4f0c19ea80 con 0x7f4f0c107a60 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.659+0000 7f4f037fe640 1 -- 172.21.15.129:0/2168543789 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4f0c19ea80 con 0x7f4f0c107a60 2024-09-05T01:06:23.965 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 msgr2=0x7f4ef803b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 0x7f4ef803b540 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0098c0 tx=0x7f4efc006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 msgr2=0x7f4f0c106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009940 tx=0x7f4f08030cc0 comp rx=0 tx=0).stop 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 shutdown_connections 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4ef8039080 0x7f4ef803b540 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 --2- 172.21.15.129:0/2168543789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f0c107a60 0x7f4f0c106030 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 >> 172.21.15.129:0/2168543789 conn(0x7f4f0c101420 msgr2=0x7f4f0c101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 shutdown_connections 2024-09-05T01:06:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:22.661+0000 7f4f143c2640 1 -- 172.21.15.129:0/2168543789 wait complete. 2024-09-05T01:06:23.969 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T01:06:23.969 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout disabled 2024-09-05T01:06:23.974 INFO:teuthology.orchestra.run.smithi129.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T01:06:23.974 INFO:teuthology.orchestra.run.smithi129.stdout:systemctl: stdout inactive 2024-09-05T01:06:23.974 INFO:teuthology.orchestra.run.smithi129.stdout:firewalld.service is not enabled 2024-09-05T01:06:23.974 INFO:teuthology.orchestra.run.smithi129.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-05T01:06:23.975 INFO:teuthology.orchestra.run.smithi129.stdout:Ceph Dashboard is now available at: 2024-09-05T01:06:23.975 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:23.976 INFO:teuthology.orchestra.run.smithi129.stdout: URL: https://smithi129:8443/ 2024-09-05T01:06:23.976 INFO:teuthology.orchestra.run.smithi129.stdout: User: admin 2024-09-05T01:06:23.976 INFO:teuthology.orchestra.run.smithi129.stdout: Password: c1nrmn0iiy 2024-09-05T01:06:23.976 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:23.976 INFO:teuthology.orchestra.run.smithi129.stdout:Saving cluster configuration to /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config directory 2024-09-05T01:06:24.963 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:24 smithi129 ceph-mon[30979]: mgrmap e13: x(active, since 6s) 2024-09-05T01:06:24.964 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:24 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1870348272' entity='client.admin' 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f723f4640 1 Processor -- start 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f723f4640 1 -- start start 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f723f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f723f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c07be10 con 0x7f7f6c108050 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39434/0 (socket says 172.21.15.129:39434) 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.347+0000 7f7f6bfff640 1 -- 172.21.15.129:0/1186933534 learned_addr learned my addr 172.21.15.129:0/1186933534 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.348+0000 7f7f6bfff640 1 -- 172.21.15.129:0/1186933534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c10a980 con 0x7f7f6c108050 2024-09-05T01:06:25.247 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.348+0000 7f7f6bfff640 1 --2- 172.21.15.129:0/1186933534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009820 tx=0x7f7f5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d18e66eaaf706 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.348+0000 7f7f6affd640 1 -- 172.21.15.129:0/1186933534 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c037070 con 0x7f7f6c108050 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f6affd640 1 -- 172.21.15.129:0/1186933534 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c0328f0 con 0x7f7f6c108050 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f6affd640 1 -- 172.21.15.129:0/1186933534 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c030740 con 0x7f7f6c108050 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 -- 172.21.15.129:0/1186933534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 msgr2=0x7f7f6c10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1186933534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009820 tx=0x7f7f5c02a270 comp rx=0 tx=0).stop 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 -- 172.21.15.129:0/1186933534 shutdown_connections 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1186933534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c10a440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 -- 172.21.15.129:0/1186933534 >> 172.21.15.129:0/1186933534 conn(0x7f7f6c101c90 msgr2=0x7f7f6c1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 -- 172.21.15.129:0/1186933534 shutdown_connections 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.349+0000 7f7f723f4640 1 -- 172.21.15.129:0/1186933534 wait complete. 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f723f4640 1 Processor -- start 2024-09-05T01:06:25.248 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f723f4640 1 -- start start 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f723f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f723f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c10b9a0 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f6bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39442/0 (socket says 172.21.15.129:39442) 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f6bfff640 1 -- 172.21.15.129:0/1870348272 learned_addr learned my addr 172.21.15.129:0/1870348272 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.350+0000 7f7f6bfff640 1 -- 172.21.15.129:0/1870348272 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c19b970 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f6bfff640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c030cc0 tx=0x7f7f5c030cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c042070 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f5c030e60 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c19bc00 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c037040 con 0x7f7f6c108050 2024-09-05T01:06:25.249 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.351+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c19e8f0 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.352+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7f5c045070 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.352+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c1084b0 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.352+0000 7f7f697fa640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 0x7f7f5403b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.352+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7f5c06c490 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.352+0000 7f7f6b7fe640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 0x7f7f5403b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.353+0000 7f7f6b7fe640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 0x7f7f5403b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f600098c0 tx=0x7f7f60006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.357+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f5c045360 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.608+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f7f6c19eac0 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.610+0000 7f7f697fa640 1 -- 172.21.15.129:0/1870348272 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f7f5c03b330 con 0x7f7f6c108050 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 msgr2=0x7f7f5403b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:25.250 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 0x7f7f5403b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f600098c0 tx=0x7f7f60006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 msgr2=0x7f7f6c19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c030cc0 tx=0x7f7f5c030cf0 comp rx=0 tx=0).stop 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 shutdown_connections 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7f54039180 0x7f7f5403b640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 --2- 172.21.15.129:0/1870348272 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f6c108050 0x7f7f6c19b430 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.612+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 >> 172.21.15.129:0/1870348272 conn(0x7f7f6c101c90 msgr2=0x7f7f6c1027c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.613+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 shutdown_connections 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:/usr/bin/ceph: stderr 2024-09-05T01:06:24.613+0000 7f7f723f4640 1 -- 172.21.15.129:0/1870348272 wait complete. 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.251 INFO:teuthology.orchestra.run.smithi129.stdout:Or, if you are only running a single cluster on this host: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: ceph telemetry on 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout:For more information see: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:25.252 INFO:teuthology.orchestra.run.smithi129.stdout:Bootstrap complete. 2024-09-05T01:06:25.283 INFO:tasks.cephadm:Fetching config... 2024-09-05T01:06:25.284 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:06:25.284 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-05T01:06:25.300 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-05T01:06:25.300 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:06:25.300 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-05T01:06:25.356 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-05T01:06:25.356 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:06:25.357 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-05T01:06:25.421 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-05T01:06:25.421 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:06:25.421 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-05T01:06:25.477 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-05T01:06:25.478 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:06:25.551 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:06:25.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:06:25.604 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:06:25.616 DEBUG:teuthology.orchestra.run.smithi194:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T01:06:25.657 INFO:teuthology.orchestra.run.smithi194.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDM4Cm/BHdhdxseL69jHfyWTZldQ1fYVKM/orHls7g5Eb1FM7s1hT1J2jTaelfqeqe3QqGieDsWJ4fE3/UClTDKrkbBlLYL3tJFhnTd9XW03GmGjcXv/DWl7h2HNbSBs8fkOBCUSuOiWVE8/UgZPrW15FV06I+RLC0gWSnK+xOVIMcl1Cb3Zl2b+6/Pz0Svv6ZTSNsHDBnehDLsJTLDuDjx7STTvGO6qJ1P4YkWXR4xZHLGb6HIZkoh3fbXsEtunjfy5FX0erVieEtPj7xc5dP/2ZZt87YPIBmHAI1XcycYOmwydIvvAqlm4C9dXmlyB/IaX9KJ4EqEYgzlWSFlP1MoMWpVV+svYmjZMK+eoe7kefoiA5odg+9vssGjeWjEQb+RPeqqLwf9WXadPRQFL+60wZUkm+7wKpXSD47RKv5XBG4wkAVDJHTJOgdpILgS7B1W+M7xJWT7dC7s76zfWTPTG5Pg4DsA+4k77NaYPXSxsh9WRRm7ngknQHEK674WuQU= ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:06:25.670 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-05T01:06:25.927 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:26.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 -- 172.21.15.129:0/1820498054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 msgr2=0x7fec4c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:26.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 --2- 172.21.15.129:0/1820498054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c100920 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009880 tx=0x7fec3c02a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 -- 172.21.15.129:0/1820498054 shutdown_connections 2024-09-05T01:06:26.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 --2- 172.21.15.129:0/1820498054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c100920 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 -- 172.21.15.129:0/1820498054 >> 172.21.15.129:0/1820498054 conn(0x7fec4c0fbcb0 msgr2=0x7fec4c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 -- 172.21.15.129:0/1820498054 shutdown_connections 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.526+0000 7fec50d00640 1 -- 172.21.15.129:0/1820498054 wait complete. 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec50d00640 1 Processor -- start 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec50d00640 1 -- start start 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec50d00640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec50d00640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1064e0 con 0x7fec4c100520 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec4a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec4a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39464/0 (socket says 172.21.15.129:39464) 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.527+0000 7fec4a59c640 1 -- 172.21.15.129:0/2821838980 learned_addr learned my addr 172.21.15.129:0/2821838980 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec4a59c640 1 -- 172.21.15.129:0/2821838980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c199a00 con 0x7fec4c100520 2024-09-05T01:06:26.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec4a59c640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009230 tx=0x7fec3c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3c03f070 con 0x7fec4c100520 2024-09-05T01:06:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c199bd0 con 0x7fec4c100520 2024-09-05T01:06:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec3c003aa0 con 0x7fec4c100520 2024-09-05T01:06:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec3c035030 con 0x7fec4c100520 2024-09-05T01:06:26.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.528+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c199f70 con 0x7fec4c100520 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.529+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fec3c042070 con 0x7fec4c100520 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.529+0000 7fec3b7fe640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 0x7fec3403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.529+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fec3c06ce50 con 0x7fec4c100520 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.529+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c106c60 con 0x7fec4c100520 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.530+0000 7fec49d9b640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 0x7fec3403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:26.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.530+0000 7fec49d9b640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 0x7fec3403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec400098c0 tx=0x7fec40006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:26.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.534+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec3c02f300 con 0x7fec4c100520 2024-09-05T01:06:26.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.703+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fec4c02d930 con 0x7fec4c100520 2024-09-05T01:06:26.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.705+0000 7fec3b7fe640 1 -- 172.21.15.129:0/2821838980 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fec3c042c60 con 0x7fec4c100520 2024-09-05T01:06:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 msgr2=0x7fec3403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 0x7fec3403b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fec400098c0 tx=0x7fec40006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 msgr2=0x7fec4c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009230 tx=0x7fec3c0048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 shutdown_connections 2024-09-05T01:06:26.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec34039060 0x7fec3403b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 --2- 172.21.15.129:0/2821838980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec4c100520 0x7fec4c06a960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:26.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.707+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 >> 172.21.15.129:0/2821838980 conn(0x7fec4c0fbcb0 msgr2=0x7fec4c0fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:26.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.708+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 shutdown_connections 2024-09-05T01:06:26.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:26.708+0000 7fec50d00640 1 -- 172.21.15.129:0/2821838980 wait complete. 2024-09-05T01:06:27.714 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-05T01:06:27.714 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2821838980' entity='client.admin' 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:27.921 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:28.152 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:28.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 -- 172.21.15.129:0/4057975931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 msgr2=0x7f5b44100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:28.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 --2- 172.21.15.129:0/4057975931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44100300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009880 tx=0x7f5b3402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 -- 172.21.15.129:0/4057975931 shutdown_connections 2024-09-05T01:06:28.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 --2- 172.21.15.129:0/4057975931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44100300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 -- 172.21.15.129:0/4057975931 >> 172.21.15.129:0/4057975931 conn(0x7f5b440fbcb0 msgr2=0x7f5b440fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:28.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 -- 172.21.15.129:0/4057975931 shutdown_connections 2024-09-05T01:06:28.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.628+0000 7f5b4a070640 1 -- 172.21.15.129:0/4057975931 wait complete. 2024-09-05T01:06:28.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b4a070640 1 Processor -- start 2024-09-05T01:06:28.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b4a070640 1 -- start start 2024-09-05T01:06:28.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b4a070640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:28.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b4a070640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5b441097e0 con 0x7f5b440fff00 2024-09-05T01:06:28.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b437fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39470/0 (socket says 172.21.15.129:39470) 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.629+0000 7f5b437fe640 1 -- 172.21.15.129:0/1579863046 learned_addr learned my addr 172.21.15.129:0/1579863046 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b437fe640 1 -- 172.21.15.129:0/1579863046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b441993f0 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b437fe640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009230 tx=0x7f5b340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b3403f070 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b34003aa0 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b34035030 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b44199680 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.630+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4419c370 con 0x7f5b440fff00 2024-09-05T01:06:28.632 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.631+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5b34042070 con 0x7f5b440fff00 2024-09-05T01:06:28.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.631+0000 7f5b40ff9640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 0x7f5b3003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:28.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.631+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5b3406dd10 con 0x7f5b440fff00 2024-09-05T01:06:28.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.631+0000 7f5b42ffd640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 0x7f5b3003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:28.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.631+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b44109f60 con 0x7f5b440fff00 2024-09-05T01:06:28.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.632+0000 7f5b42ffd640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 0x7f5b3003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098c0 tx=0x7f5b2c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:28.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.636+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b34039350 con 0x7f5b440fff00 2024-09-05T01:06:28.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.810+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5b44062630 con 0x7f5b30039060 2024-09-05T01:06:28.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.813+0000 7f5b40ff9640 1 -- 172.21.15.129:0/1579863046 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5b44062630 con 0x7f5b30039060 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 msgr2=0x7f5b3003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 0x7f5b3003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0098c0 tx=0x7f5b2c006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 msgr2=0x7f5b44198eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009230 tx=0x7f5b340048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 shutdown_connections 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5b30039060 0x7f5b3003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 --2- 172.21.15.129:0/1579863046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5b440fff00 0x7f5b44198eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 >> 172.21.15.129:0/1579863046 conn(0x7f5b440fbcb0 msgr2=0x7f5b440fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 shutdown_connections 2024-09-05T01:06:28.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:28.816+0000 7f5b4a070640 1 -- 172.21.15.129:0/1579863046 wait complete. 2024-09-05T01:06:29.564 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-09-05T01:06:29.564 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:06:29.564 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-09-05T01:06:29.584 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:06:29.584 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:29.642 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-09-05T01:06:29.642 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch host add smithi183 2024-09-05T01:06:30.072 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:30.193 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:30.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:29 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:30.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 -- 172.21.15.129:0/1332085416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 msgr2=0x7fbe84101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:30.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 --2- 172.21.15.129:0/1332085416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84101650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009880 tx=0x7fbe7402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:30.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 -- 172.21.15.129:0/1332085416 shutdown_connections 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 --2- 172.21.15.129:0/1332085416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84101650 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 -- 172.21.15.129:0/1332085416 >> 172.21.15.129:0/1332085416 conn(0x7fbe840fb500 msgr2=0x7fbe840fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 -- 172.21.15.129:0/1332085416 shutdown_connections 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.510+0000 7fbe8884c640 1 -- 172.21.15.129:0/1332085416 wait complete. 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.511+0000 7fbe8884c640 1 Processor -- start 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.511+0000 7fbe8884c640 1 -- start start 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.511+0000 7fbe8884c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.511+0000 7fbe8884c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8410aa40 con 0x7fbe84101270 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39488/0 (socket says 172.21.15.129:39488) 2024-09-05T01:06:30.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8259c640 1 -- 172.21.15.129:0/889816157 learned_addr learned my addr 172.21.15.129:0/889816157 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8259c640 1 -- 172.21.15.129:0/889816157 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe84197580 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8259c640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009230 tx=0x7fbe740048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7403f070 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe84197870 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe74004d10 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe74035030 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.512+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8419b910 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.513+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fbe74042070 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.513+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe840ffae0 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.513+0000 7fbe737fe640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 0x7fbe6c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.513+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbe7406cc40 con 0x7fbe84101270 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.513+0000 7fbe81d9b640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 0x7fbe6c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:30.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.514+0000 7fbe81d9b640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 0x7fbe6c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe780098c0 tx=0x7fbe78006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:30.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.518+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe74042c60 con 0x7fbe84101270 2024-09-05T01:06:30.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:30.689+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}) -- 0x7fbe841983a0 con 0x7fbe6c039060 2024-09-05T01:06:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:30 smithi129 ceph-mon[30979]: Updating smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:06:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:30 smithi129 ceph-mon[30979]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:31.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:30 smithi129 ceph-mon[30979]: Updating smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:06:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:31 smithi129 ceph-mon[30979]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:32 smithi129 ceph-mon[30979]: Deploying cephadm binary to smithi183 2024-09-05T01:06:33.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.781+0000 7fbe737fe640 1 -- 172.21.15.129:0/889816157 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbe841983a0 con 0x7fbe6c039060 2024-09-05T01:06:33.782 INFO:teuthology.orchestra.run.smithi129.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 msgr2=0x7fbe6c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 0x7fbe6c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbe780098c0 tx=0x7fbe78006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 msgr2=0x7fbe84197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009230 tx=0x7fbe740048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 shutdown_connections 2024-09-05T01:06:33.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbe6c039060 0x7fbe6c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:33.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 --2- 172.21.15.129:0/889816157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe84101270 0x7fbe84197040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:33.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 >> 172.21.15.129:0/889816157 conn(0x7fbe840fb500 msgr2=0x7fbe840fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:33.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 shutdown_connections 2024-09-05T01:06:33.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:33.783+0000 7fbe8884c640 1 -- 172.21.15.129:0/889816157 wait complete. 2024-09-05T01:06:34.402 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T01:06:34.645 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:35.052 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:35.053 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:34 smithi129 ceph-mon[30979]: Added host smithi183 2024-09-05T01:06:35.053 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:35.053 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:35.053 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1903248704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 msgr2=0x7f5cd4102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1903248704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd4102b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009880 tx=0x7f5cc402a670 comp rx=0 tx=0).stop 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1903248704 shutdown_connections 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1903248704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd4102b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1903248704 >> 172.21.15.129:0/1903248704 conn(0x7f5cd40fc1a0 msgr2=0x7f5cd40fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1903248704 shutdown_connections 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.132+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1903248704 wait complete. 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd96fd640 1 Processor -- start 2024-09-05T01:06:35.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd96fd640 1 -- start start 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd96fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd96fd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410bba0 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59178/0 (socket says 172.21.15.129:59178) 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.133+0000 7f5cd2ffd640 1 -- 172.21.15.129:0/1962718824 learned_addr learned my addr 172.21.15.129:0/1962718824 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cd2ffd640 1 -- 172.21.15.129:0/1962718824 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd4197610 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cd2ffd640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009230 tx=0x7f5cc40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc403f070 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cc4004d10 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41978a0 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cc4035030 con 0x7f5cd4102780 2024-09-05T01:06:35.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.134+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd4197d00 con 0x7f5cd4102780 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.135+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f5cc4042070 con 0x7f5cd4102780 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.135+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd406a7c0 con 0x7f5cd4102780 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.135+0000 7f5cbbfff640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.135+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cc406ccb0 con 0x7f5cd4102780 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.135+0000 7f5cd27fc640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.136+0000 7f5cd27fc640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 0x7f5cbc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80098c0 tx=0x7f5cc8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:35.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.140+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cc4039350 con 0x7f5cd4102780 2024-09-05T01:06:35.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.320+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cd4198430 con 0x7f5cbc0390c0 2024-09-05T01:06:35.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.321+0000 7f5cbbfff640 1 -- 172.21.15.129:0/1962718824 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f5cd4198430 con 0x7f5cbc0390c0 2024-09-05T01:06:35.321 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:35.322 INFO:teuthology.orchestra.run.smithi129.stdout:[{"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 msgr2=0x7f5cbc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 0x7f5cbc03b580 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5cc80098c0 tx=0x7f5cc8006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 msgr2=0x7f5cd41970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4009230 tx=0x7f5cc40048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 shutdown_connections 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5cbc0390c0 0x7f5cbc03b580 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 --2- 172.21.15.129:0/1962718824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd4102780 0x7f5cd41970d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:35.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 >> 172.21.15.129:0/1962718824 conn(0x7f5cd40fc1a0 msgr2=0x7f5cd4105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:35.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 shutdown_connections 2024-09-05T01:06:35.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:35.323+0000 7f5cd96fd640 1 -- 172.21.15.129:0/1962718824 wait complete. 2024-09-05T01:06:35.975 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi194 2024-09-05T01:06:35.976 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:06:35.976 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.conf 2024-09-05T01:06:35.995 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:06:35.995 DEBUG:teuthology.orchestra.run.smithi194:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:36.054 INFO:tasks.cephadm:Adding host smithi194 to orchestrator... 2024-09-05T01:06:36.054 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch host add smithi194 2024-09-05T01:06:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:36 smithi129 ceph-mon[30979]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:06:36.304 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:36.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 -- 172.21.15.129:0/1003554013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 msgr2=0x7fe8d8106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:36.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 --2- 172.21.15.129:0/1003554013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8106970 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009880 tx=0x7fe8d402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:36.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 -- 172.21.15.129:0/1003554013 shutdown_connections 2024-09-05T01:06:36.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 --2- 172.21.15.129:0/1003554013 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8106970 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 -- 172.21.15.129:0/1003554013 >> 172.21.15.129:0/1003554013 conn(0x7fe8d80fc1a0 msgr2=0x7fe8d80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 -- 172.21.15.129:0/1003554013 shutdown_connections 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.722+0000 7fe8df80b640 1 -- 172.21.15.129:0/1003554013 wait complete. 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8df80b640 1 Processor -- start 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8df80b640 1 -- start start 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8df80b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8df80b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d810aa70 con 0x7fe8d8106590 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8dd5a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8dd5a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59204/0 (socket says 172.21.15.129:59204) 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8dd5a7640 1 -- 172.21.15.129:0/3815928858 learned_addr learned my addr 172.21.15.129:0/3815928858 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:36.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8dd5a7640 1 -- 172.21.15.129:0/3815928858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d8198420 con 0x7fe8d8106590 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8dd5a7640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009230 tx=0x7fe8d40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.723+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d403f070 con 0x7fe8d8106590 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.724+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d81986b0 con 0x7fe8d8106590 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.724+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8d4004d10 con 0x7fe8d8106590 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.724+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8d4035030 con 0x7fe8d8106590 2024-09-05T01:06:36.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.724+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d8198b10 con 0x7fe8d8106590 2024-09-05T01:06:36.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.724+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe8d4042070 con 0x7fe8d8106590 2024-09-05T01:06:36.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.725+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d8101c00 con 0x7fe8d8106590 2024-09-05T01:06:36.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.725+0000 7fe8ce7fc640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 0x7fe8c403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:36.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.725+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8d406ccf0 con 0x7fe8d8106590 2024-09-05T01:06:36.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.725+0000 7fe8dcda6640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 0x7fe8c403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:36.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.725+0000 7fe8dcda6640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 0x7fe8c403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098c0 tx=0x7fe8c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:36.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.729+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8d4071070 con 0x7fe8d8106590 2024-09-05T01:06:36.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:36.897+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}) -- 0x7fe8d81069f0 con 0x7fe8c4039060 2024-09-05T01:06:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi194", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: Deploying cephadm binary to smithi194 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:38.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:06:39.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-05T01:06:39.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: Updating smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:06:39.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:39.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: Updating smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:06:39.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:39.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:39.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:40.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.020+0000 7fe8ce7fc640 1 -- 172.21.15.129:0/3815928858 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe8d81069f0 con 0x7fe8c4039060 2024-09-05T01:06:40.021 INFO:teuthology.orchestra.run.smithi129.stdout:Added host 'smithi194' with addr '172.21.15.194' 2024-09-05T01:06:40.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 msgr2=0x7fe8c403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:40.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 0x7fe8c403b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80098c0 tx=0x7fe8c8006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:40.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 msgr2=0x7fe8d8197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:40.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe8d4009230 tx=0x7fe8d40048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 shutdown_connections 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe8c4039060 0x7fe8c403b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 --2- 172.21.15.129:0/3815928858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8d8106590 0x7fe8d8197ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.022+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 >> 172.21.15.129:0/3815928858 conn(0x7fe8d80fc1a0 msgr2=0x7fe8d80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.023+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 shutdown_connections 2024-09-05T01:06:40.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:40.023+0000 7fe8df80b640 1 -- 172.21.15.129:0/3815928858 wait complete. 2024-09-05T01:06:40.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T01:06:40.887 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:40 smithi129 ceph-mon[30979]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.368+0000 7f8c28e38640 1 -- 172.21.15.129:0/2734391573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 msgr2=0x7f8c24107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.368+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2734391573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24107a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009880 tx=0x7f8c1802a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.368+0000 7f8c28e38640 1 -- 172.21.15.129:0/2734391573 shutdown_connections 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.368+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2734391573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24107a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.368+0000 7f8c28e38640 1 -- 172.21.15.129:0/2734391573 >> 172.21.15.129:0/2734391573 conn(0x7f8c24076250 msgr2=0x7f8c24076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 -- 172.21.15.129:0/2734391573 shutdown_connections 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 -- 172.21.15.129:0/2734391573 wait complete. 2024-09-05T01:06:41.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 Processor -- start 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 -- start start 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.369+0000 7f8c28e38640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2410bb90 con 0x7f8c24107690 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59228/0 (socket says 172.21.15.129:59228) 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c2259c640 1 -- 172.21.15.129:0/2699886786 learned_addr learned my addr 172.21.15.129:0/2699886786 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c2259c640 1 -- 172.21.15.129:0/2699886786 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c24199880 con 0x7f8c24107690 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c2259c640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009230 tx=0x7f8c180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c1803f070 con 0x7f8c24107690 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c18004d10 con 0x7f8c24107690 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c24199b10 con 0x7f8c24107690 2024-09-05T01:06:41.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c18035030 con 0x7f8c24107690 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.370+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c24199f70 con 0x7f8c24107690 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.371+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8c18042070 con 0x7f8c24107690 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.371+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c24102d30 con 0x7f8c24107690 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.371+0000 7f8c137fe640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 0x7f8c0c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.371+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8c1806ccd0 con 0x7f8c24107690 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.371+0000 7f8c21d9b640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 0x7f8c0c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:41.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.372+0000 7f8c21d9b640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 0x7f8c0c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c140098c0 tx=0x7f8c14006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:41.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.376+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c18042360 con 0x7f8c24107690 2024-09-05T01:06:41.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.546+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c2419a6a0 con 0x7f8c0c039060 2024-09-05T01:06:41.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.547+0000 7f8c137fe640 1 -- 172.21.15.129:0/2699886786 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f8c2419a6a0 con 0x7f8c0c039060 2024-09-05T01:06:41.548 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:06:41.548 INFO:teuthology.orchestra.run.smithi129.stdout:[{"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.194", "hostname": "smithi194", "labels": [], "status": ""}] 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 msgr2=0x7f8c0c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 0x7f8c0c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c140098c0 tx=0x7f8c14006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 msgr2=0x7f8c24199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c18009230 tx=0x7f8c180048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 shutdown_connections 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c0c039060 0x7f8c0c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 --2- 172.21.15.129:0/2699886786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c24107690 0x7f8c24199340 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 >> 172.21.15.129:0/2699886786 conn(0x7f8c24076250 msgr2=0x7f8c241039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:41.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 shutdown_connections 2024-09-05T01:06:41.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:41.549+0000 7f8c28e38640 1 -- 172.21.15.129:0/2699886786 wait complete. 2024-09-05T01:06:41.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:41 smithi129 ceph-mon[30979]: Added host smithi194 2024-09-05T01:06:41.892 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:42.198 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-05T01:06:42.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd crush tunables default 2024-09-05T01:06:42.438 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:06:42.809 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:42 smithi129 ceph-mon[30979]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:06:42.809 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:42 smithi129 ceph-mon[30979]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:42.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 -- 172.21.15.129:0/2106013103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 msgr2=0x7f3724102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:42.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 --2- 172.21.15.129:0/2106013103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724102b60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3714009880 tx=0x7f371402a670 comp rx=0 tx=0).stop 2024-09-05T01:06:42.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 -- 172.21.15.129:0/2106013103 shutdown_connections 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 --2- 172.21.15.129:0/2106013103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724102b60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 -- 172.21.15.129:0/2106013103 >> 172.21.15.129:0/2106013103 conn(0x7f37240fc1a0 msgr2=0x7f37240fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 -- 172.21.15.129:0/2106013103 shutdown_connections 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.884+0000 7f3728ff1640 1 -- 172.21.15.129:0/2106013103 wait complete. 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.885+0000 7f3728ff1640 1 Processor -- start 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.885+0000 7f3728ff1640 1 -- start start 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.885+0000 7f3728ff1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.885+0000 7f3728ff1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f372410bba0 con 0x7f3724102780 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.885+0000 7f372259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f372259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36770/0 (socket says 172.21.15.129:36770) 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f372259c640 1 -- 172.21.15.129:0/4249227146 learned_addr learned my addr 172.21.15.129:0/4249227146 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:06:42.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f372259c640 1 -- 172.21.15.129:0/4249227146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3724197690 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f372259c640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f37140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f371403f070 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3714004d10 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3724197920 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3714035030 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.886+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3724197d80 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.887+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3714042070 con 0x7f3724102780 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.887+0000 7f371bfff640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 0x7f371003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:42.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.887+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f371406dba0 con 0x7f3724102780 2024-09-05T01:06:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.887+0000 7f3719d9b640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 0x7f371003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.887+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372406a7c0 con 0x7f3724102780 2024-09-05T01:06:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.888+0000 7f3719d9b640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 0x7f371003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f370c0098c0 tx=0x7f370c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:42.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:42.892+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3714039350 con 0x7f3724102780 2024-09-05T01:06:43.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.062+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f37241985c0 con 0x7f3724102780 2024-09-05T01:06:43.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.677+0000 7f371bfff640 1 -- 172.21.15.129:0/4249227146 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f371402f300 con 0x7f3724102780 2024-09-05T01:06:43.678 INFO:teuthology.orchestra.run.smithi129.stderr:adjusted tunables profile to default 2024-09-05T01:06:43.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.680+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 msgr2=0x7f371003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:43.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.680+0000 7f3728ff1640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 0x7f371003b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f370c0098c0 tx=0x7f370c006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.680+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 msgr2=0x7f3724197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.680+0000 7f3728ff1640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3714009230 tx=0x7f37140048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 shutdown_connections 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3710039060 0x7f371003b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 --2- 172.21.15.129:0/4249227146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3724102780 0x7f3724197150 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 >> 172.21.15.129:0/4249227146 conn(0x7f37240fc1a0 msgr2=0x7f3724105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 shutdown_connections 2024-09-05T01:06:43.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:06:43.681+0000 7f3728ff1640 1 -- 172.21.15.129:0/4249227146 wait complete. 2024-09-05T01:06:43.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:43 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4249227146' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T01:06:44.317 INFO:tasks.cephadm:Adding mon.a on smithi129 2024-09-05T01:06:44.317 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-09-05T01:06:44.318 INFO:tasks.cephadm:Adding mon.c on smithi194 2024-09-05T01:06:44.318 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch apply mon '3;smithi129:172.21.15.129=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c' 2024-09-05T01:06:44.561 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4249227146' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:44.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:44.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi194", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:06:44.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:44.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 -- 172.21.15.194:0/4103201168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 msgr2=0x7fe930100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 --2- 172.21.15.194:0/4103201168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930100d70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe920009880 tx=0x7fe92002a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 -- 172.21.15.194:0/4103201168 shutdown_connections 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 --2- 172.21.15.194:0/4103201168 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930100d70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 -- 172.21.15.194:0/4103201168 >> 172.21.15.194:0/4103201168 conn(0x7fe930076250 msgr2=0x7fe930076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 -- 172.21.15.194:0/4103201168 shutdown_connections 2024-09-05T01:06:45.069 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.068+0000 7fe93525e640 1 -- 172.21.15.194:0/4103201168 wait complete. 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe93525e640 1 Processor -- start 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe93525e640 1 -- start start 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe93525e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe93525e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010aa40 con 0x7fe930100990 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe92ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.069+0000 7fe92ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.194:42018/0 (socket says 172.21.15.194:42018) 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92ed9d640 1 -- 172.21.15.194:0/2766902570 learned_addr learned my addr 172.21.15.194:0/2766902570 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92ed9d640 1 -- 172.21.15.194:0/2766902570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe930104080 con 0x7fe930100990 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92ed9d640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe920009230 tx=0x7fe9200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92003f070 con 0x7fe930100990 2024-09-05T01:06:45.071 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe920004d10 con 0x7fe930100990 2024-09-05T01:06:45.072 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe930102730 con 0x7fe930100990 2024-09-05T01:06:45.072 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe920035030 con 0x7fe930100990 2024-09-05T01:06:45.072 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.070+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe930102bf0 con 0x7fe930100990 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.071+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe920042070 con 0x7fe930100990 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.071+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930101c40 con 0x7fe930100990 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.071+0000 7fe92cd99640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 0x7fe91c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.071+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe92006dbc0 con 0x7fe930100990 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.072+0000 7fe927fff640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 0x7fe91c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:45.073 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.072+0000 7fe927fff640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 0x7fe91c03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9180098c0 tx=0x7fe918006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:45.077 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.076+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe920039350 con 0x7fe930100990 2024-09-05T01:06:45.261 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.260+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi129:172.21.15.129=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}) -- 0x7fe930103780 con 0x7fe91c0390c0 2024-09-05T01:06:45.266 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.266+0000 7fe92cd99640 1 -- 172.21.15.194:0/2766902570 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe930103780 con 0x7fe91c0390c0 2024-09-05T01:06:45.267 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mon update... 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 msgr2=0x7fe91c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 0x7fe91c03b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9180098c0 tx=0x7fe918006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 msgr2=0x7fe930102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe920009230 tx=0x7fe9200048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 shutdown_connections 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe91c0390c0 0x7fe91c03b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 --2- 172.21.15.194:0/2766902570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe930100990 0x7fe930102190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 >> 172.21.15.194:0/2766902570 conn(0x7fe930076250 msgr2=0x7fe9301042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 shutdown_connections 2024-09-05T01:06:45.269 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:45.268+0000 7fe93525e640 1 -- 172.21.15.194:0/2766902570 wait complete. 2024-09-05T01:06:45.875 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.b.service 2024-09-05T01:06:45.878 DEBUG:teuthology.orchestra.run.smithi194:mon.c> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.c.service 2024-09-05T01:06:45.881 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:06:45.881 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:06:45.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: Updating smithi194:/etc/ceph/ceph.conf 2024-09-05T01:06:45.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: Updating smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:06:45.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: Updating smithi194:/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: Updating smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:06:45.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:46.654 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:06:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:46 smithi129 ceph-mon[30979]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi129:172.21.15.129=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:06:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:46 smithi129 ceph-mon[30979]: Saving service mon spec with placement smithi129:172.21.15.129=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c;count:3 2024-09-05T01:06:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:46 smithi129 ceph-mon[30979]: Deploying daemon mon.c on smithi194 2024-09-05T01:06:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:46 smithi129 ceph-mon[30979]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:47.082 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 -- 172.21.15.194:0/1747105889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 msgr2=0x7f498c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 --2- 172.21.15.194:0/1747105889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c106970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f497c009880 tx=0x7f497c02a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 -- 172.21.15.194:0/1747105889 shutdown_connections 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 --2- 172.21.15.194:0/1747105889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c106970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 -- 172.21.15.194:0/1747105889 >> 172.21.15.194:0/1747105889 conn(0x7f498c0fc1a0 msgr2=0x7f498c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 -- 172.21.15.194:0/1747105889 shutdown_connections 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 -- 172.21.15.194:0/1747105889 wait complete. 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.065+0000 7f4991e69640 1 Processor -- start 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f4991e69640 1 -- start start 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f4991e69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f4991e69640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f498c109960 con 0x7f498c106590 2024-09-05T01:06:47.083 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f498b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.194:42036/0 (socket says 172.21.15.194:42036) 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.066+0000 7f498b7fe640 1 -- 172.21.15.194:0/1359471020 learned_addr learned my addr 172.21.15.194:0/1359471020 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f498b7fe640 1 -- 172.21.15.194:0/1359471020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c1953a0 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f498b7fe640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f497c009230 tx=0x7f497c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c03f070 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f497c004d10 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c195630 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497c035030 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.067+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c195a90 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.068+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f497c042070 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.068+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f498c101c00 con 0x7f498c106590 2024-09-05T01:06:47.084 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.068+0000 7f4988ff9640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 0x7f497403d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:47.085 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.068+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f497c06cd30 con 0x7f498c106590 2024-09-05T01:06:47.085 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.068+0000 7f498affd640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 0x7f497403d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:47.085 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.069+0000 7f498affd640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 0x7f497403d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49800098c0 tx=0x7f4980006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:47.085 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.073+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f497c0372d0 con 0x7f498c106590 2024-09-05T01:06:47.310 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.310+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f498c196220 con 0x7f498c106590 2024-09-05T01:06:47.311 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.310+0000 7f4988ff9640 1 -- 172.21.15.194:0/1359471020 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f497c0055a0 con 0x7f498c106590 2024-09-05T01:06:47.311 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:06:47.311 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:05:29.598613Z","created":"2024-09-05T01:05:29.598613Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T01:06:47.311 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-09-05T01:06:47.313 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 msgr2=0x7f497403d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:47.313 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 0x7f497403d780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49800098c0 tx=0x7f4980006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.313 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 msgr2=0x7f498c194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:47.313 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f497c009230 tx=0x7f497c0048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 shutdown_connections 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f497403b2c0 0x7f497403d780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 --2- 172.21.15.194:0/1359471020 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f498c106590 0x7f498c194e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.312+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 >> 172.21.15.194:0/1359471020 conn(0x7f498c0fc1a0 msgr2=0x7f498c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.313+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 shutdown_connections 2024-09-05T01:06:47.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:47.313+0000 7f4991e69640 1 -- 172.21.15.194:0/1359471020 wait complete. 2024-09-05T01:06:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/1359471020' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:48.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:48 smithi129 ceph-mon[30979]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:49.700 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:06:49.700 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:06:50.045 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 -- 172.21.15.194:0/4034010925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 msgr2=0x7f2758102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 --2- 172.21.15.194:0/4034010925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f2758102c00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f274c009880 tx=0x7f274c02a670 comp rx=0 tx=0).stop 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 -- 172.21.15.194:0/4034010925 shutdown_connections 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 --2- 172.21.15.194:0/4034010925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f2758102c00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 -- 172.21.15.194:0/4034010925 >> 172.21.15.194:0/4034010925 conn(0x7f27580fc220 msgr2=0x7f27580fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 -- 172.21.15.194:0/4034010925 shutdown_connections 2024-09-05T01:06:50.438 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.434+0000 7f275ed72640 1 -- 172.21.15.194:0/4034010925 wait complete. 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275ed72640 1 Processor -- start 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275ed72640 1 -- start start 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275ed72640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275ed72640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f275810bc20 con 0x7f2758102820 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275cb0e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275cb0e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.194:44014/0 (socket says 172.21.15.194:44014) 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275cb0e640 1 -- 172.21.15.194:0/835004334 learned_addr learned my addr 172.21.15.194:0/835004334 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.435+0000 7f275cb0e640 1 -- 172.21.15.194:0/835004334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275819b2d0 con 0x7f2758102820 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f275cb0e640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f274c009230 tx=0x7f274c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f274c03f070 con 0x7f2758102820 2024-09-05T01:06:50.439 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f274c004d10 con 0x7f2758102820 2024-09-05T01:06:50.440 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275819b560 con 0x7f2758102820 2024-09-05T01:06:50.440 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f274c035030 con 0x7f2758102820 2024-09-05T01:06:50.440 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.436+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2758194fe0 con 0x7f2758102820 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.437+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f274c042070 con 0x7f2758102820 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.437+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275806ad10 con 0x7f2758102820 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.437+0000 7f2755ffb640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 0x7f274803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.437+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f274c06cd20 con 0x7f2758102820 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.437+0000 7f2757fff640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 0x7f274803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:50.441 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.438+0000 7f2757fff640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 0x7f274803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27440098c0 tx=0x7f2744006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:50.444 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.442+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f274c039350 con 0x7f2758102820 2024-09-05T01:06:50.681 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.679+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2758196340 con 0x7f2758102820 2024-09-05T01:06:50.681 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.679+0000 7f2755ffb640 1 -- 172.21.15.194:0/835004334 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7f274c02f300 con 0x7f2758102820 2024-09-05T01:06:50.682 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:06:50.682 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":1,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:05:29.598613Z","created":"2024-09-05T01:05:29.598613Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T01:06:50.682 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 1 2024-09-05T01:06:50.683 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 msgr2=0x7f274803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 0x7f274803b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f27440098c0 tx=0x7f2744006db0 comp rx=0 tx=0).stop 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 msgr2=0x7f275819ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f274c009230 tx=0x7f274c0048f0 comp rx=0 tx=0).stop 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 shutdown_connections 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2748039060 0x7f274803b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 --2- 172.21.15.194:0/835004334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2758102820 0x7f275819ad90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 >> 172.21.15.194:0/835004334 conn(0x7f27580fc220 msgr2=0x7f27581003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 shutdown_connections 2024-09-05T01:06:50.684 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:50.682+0000 7f275ed72640 1 -- 172.21.15.194:0/835004334 wait complete. 2024-09-05T01:06:50.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:50 smithi129 ceph-mon[30979]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:50.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:50 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/835004334' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:52.539 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:06:52.539 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:06:52.810 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:06:56.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:56.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: mon.a calling monitor election 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: mon.c calling monitor election 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:06:56.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: monmap epoch 2 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: last_changed 2024-09-05T01:06:50.855083+0000 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: election_strategy: 1 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: fsmap 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: mgrmap e13: x(active, since 38s) 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:06:56.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:56.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:56.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:06:56.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:06:56.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:06:56.252 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 -- 172.21.15.194:0/1733480832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 msgr2=0x7ff2a40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/1733480832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2a40059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff2b806a240 tx=0x7ff2a802bce0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 -- 172.21.15.194:0/1733480832 shutdown_connections 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/1733480832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2a40059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 -- 172.21.15.194:0/1733480832 >> 172.21.15.194:0/1733480832 conn(0x7ff2b80fc280 msgr2=0x7ff2b80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 -- 172.21.15.194:0/1733480832 shutdown_connections 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.250+0000 7ff2bfb44640 1 -- 172.21.15.194:0/1733480832 wait complete. 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 Processor -- start 2024-09-05T01:06:56.253 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 -- start start 2024-09-05T01:06:56.255 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2b81004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8109650 con 0x7ff2a4003530 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bfb44640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b81094d0 con 0x7ff2b81009e0 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bd0df640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bd0df640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:50916/0 (socket says 172.21.15.194:50916) 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.251+0000 7ff2bd0df640 1 -- 172.21.15.194:0/2824925191 learned_addr learned my addr 172.21.15.194:0/2824925191 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd8e0640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2b81004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd0df640 1 -- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 msgr2=0x7ff2b8100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd0df640 1 -- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 msgr2=0x7ff2b8100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd0df640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd0df640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:06:56.256 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd8e0640 1 -- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 msgr2=0x7ff2b8100e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd8e0640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd8e0640 1 -- 172.21.15.194:0/2824925191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b8102390 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.252+0000 7ff2bd8e0640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2b81004a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802c2d0 tx=0x7ff2a802d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.253+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff2a80035f0 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.253+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2a8003c40 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.253+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b81a34b0 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.253+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff2a8037cb0 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.253+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b81a3a40 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.254+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff2a804a440 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.254+0000 7ff2aeffd640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 0x7ff29c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.254+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff2a806e760 con 0x7ff2a4003530 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.254+0000 7ff2bd0df640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 0x7ff29c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:56.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.254+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b806a4f0 con 0x7ff2a4003530 2024-09-05T01:06:56.258 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.255+0000 7ff2bd0df640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 0x7ff29c03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009100 tx=0x7ff2b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:56.261 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.259+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2a8048e20 con 0x7ff2a4003530 2024-09-05T01:06:56.501 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.499+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff2b81a3cf0 con 0x7ff2a4003530 2024-09-05T01:06:56.501 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.500+0000 7ff2aeffd640 1 -- 172.21.15.194:0/2824925191 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7ff2a804f020 con 0x7ff2a4003530 2024-09-05T01:06:56.502 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:06:56.502 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:06:50.855083Z","created":"2024-09-05T01:05:29.598613Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:06:56.502 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-09-05T01:06:56.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 msgr2=0x7ff29c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:56.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 0x7ff29c03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009100 tx=0x7ff2b4008040 comp rx=0 tx=0).stop 2024-09-05T01:06:56.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 msgr2=0x7ff2b81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:56.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2b81004a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff2a802c2d0 tx=0x7ff2a802d990 comp rx=0 tx=0).stop 2024-09-05T01:06:56.504 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 shutdown_connections 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff29c038e50 0x7ff29c03b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff2b81009e0 0x7ff2b8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 --2- 172.21.15.194:0/2824925191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2a4003530 0x7ff2b81004a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 >> 172.21.15.194:0/2824925191 conn(0x7ff2b80fc280 msgr2=0x7ff2b818fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 shutdown_connections 2024-09-05T01:06:56.505 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:56.502+0000 7ff2bfb44640 1 -- 172.21.15.194:0/2824925191 wait complete. 2024-09-05T01:06:57.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:56 smithi194 ceph-mon[34941]: Deploying daemon mon.b on smithi183 2024-09-05T01:06:57.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:56 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/2824925191' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:57.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:56 smithi129 ceph-mon[30979]: Deploying daemon mon.b on smithi183 2024-09-05T01:06:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:56 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/2824925191' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:06:58.143 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:06:58.143 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:06:58.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:57 smithi129 ceph-mon[30979]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:57 smithi194 ceph-mon[34941]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:06:58.399 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:06:58.807 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- 172.21.15.194:0/3716666057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f4107670 msgr2=0x7fe4f4107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 --2- 172.21.15.194:0/3716666057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f4107670 0x7fe4f4107a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009880 tx=0x7fe4f002a5d0 comp rx=0 tx=0).stop 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- 172.21.15.194:0/3716666057 shutdown_connections 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 --2- 172.21.15.194:0/3716666057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f4107670 0x7fe4f4107a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- 172.21.15.194:0/3716666057 >> 172.21.15.194:0/3716666057 conn(0x7fe4f40fd330 msgr2=0x7fe4f40ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- 172.21.15.194:0/3716666057 shutdown_connections 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- 172.21.15.194:0/3716666057 wait complete. 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 Processor -- start 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.804+0000 7fe4fc789640 1 -- start start 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fc789640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fc789640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 0x7fe4f419bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fc789640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f410bbe0 con 0x7fe4f41977b0 2024-09-05T01:06:58.808 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fc789640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f410ba60 con 0x7fe4f4107670 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:37658/0 (socket says 172.21.15.194:37658) 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 -- 172.21.15.194:0/384765044 learned_addr learned my addr 172.21.15.194:0/384765044 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4f9d24640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 0x7fe4f419bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 -- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 msgr2=0x7fe4f4197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 -- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 msgr2=0x7fe4f4197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4fa525640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4f9d24640 1 -- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 msgr2=0x7fe4f4197270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4f9d24640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:58.809 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.805+0000 7fe4f9d24640 1 -- 172.21.15.194:0/384765044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f4197dd0 con 0x7fe4f41977b0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4f9d24640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 0x7fe4f419bb40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b3d0 tx=0x7fe4e400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe4e4011630 con 0x7fe4f41977b0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f419c080 con 0x7fe4f41977b0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4e4011c80 con 0x7fe4f41977b0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe4e4010d80 con 0x7fe4f41977b0 2024-09-05T01:06:58.810 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.806+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f419c5c0 con 0x7fe4f41977b0 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.807+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe4e4018020 con 0x7fe4f41977b0 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.807+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f4102db0 con 0x7fe4f41977b0 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.807+0000 7fe4eb7fe640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 0x7fe4e003b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.807+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe4e404bcf0 con 0x7fe4f41977b0 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.807+0000 7fe4fa525640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 0x7fe4e003b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:06:58.811 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.808+0000 7fe4fa525640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 0x7fe4e003b2b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4f002abc0 tx=0x7fe4f002aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:06:58.814 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:58.813+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4e401f020 con 0x7fe4f41977b0 2024-09-05T01:06:59.041 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.040+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe4f419c8d0 con 0x7fe4f41977b0 2024-09-05T01:06:59.042 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.040+0000 7fe4eb7fe640 1 -- 172.21.15.194:0/384765044 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fe4e4017dd0 con 0x7fe4f41977b0 2024-09-05T01:06:59.042 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:06:59.042 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":2,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:06:50.855083Z","created":"2024-09-05T01:05:29.598613Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:06:59.043 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 2 2024-09-05T01:06:59.044 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 msgr2=0x7fe4e003b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:59.044 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 0x7fe4e003b2b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4f002abc0 tx=0x7fe4f002aad0 comp rx=0 tx=0).stop 2024-09-05T01:06:59.044 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 msgr2=0x7fe4f419bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:06:59.044 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 0x7fe4f419bb40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400b3d0 tx=0x7fe4e400b890 comp rx=0 tx=0).stop 2024-09-05T01:06:59.044 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 shutdown_connections 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4e0038df0 0x7fe4e003b2b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f41977b0 0x7fe4f419bb40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 --2- 172.21.15.194:0/384765044 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4f4107670 0x7fe4f4197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 >> 172.21.15.194:0/384765044 conn(0x7fe4f40fd330 msgr2=0x7fe4f40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 shutdown_connections 2024-09-05T01:06:59.045 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:06:59.042+0000 7fe4fc789640 1 -- 172.21.15.194:0/384765044 wait complete. 2024-09-05T01:06:59.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:06:59 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/384765044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:59.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:06:59 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/384765044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:06:59.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:06:59 smithi183 systemd[1]: Started Ceph mon.b for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:07:00.600 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T01:07:00.601 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T01:07:00.850 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:07:04.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 -- 172.21.15.194:0/1741085880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f880c003530 msgr2=0x7f880c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:04.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 --2- 172.21.15.194:0/1741085880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f880c003530 0x7f880c0059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f88241016a0 tx=0x7f881402c3f0 comp rx=0 tx=0).stop 2024-09-05T01:07:04.898 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 -- 172.21.15.194:0/1741085880 shutdown_connections 2024-09-05T01:07:04.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 --2- 172.21.15.194:0/1741085880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f880c003530 0x7f880c0059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:04.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 -- 172.21.15.194:0/1741085880 >> 172.21.15.194:0/1741085880 conn(0x7f88240fc220 msgr2=0x7f88240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:04.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 -- 172.21.15.194:0/1741085880 shutdown_connections 2024-09-05T01:07:04.899 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.897+0000 7f8829058640 1 -- 172.21.15.194:0/1741085880 wait complete. 2024-09-05T01:07:04.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.898+0000 7f8829058640 1 Processor -- start 2024-09-05T01:07:04.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 -- start start 2024-09-05T01:07:04.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:04.901 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 0x7f882419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8824108520 con 0x7f88241977d0 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88241083a0 con 0x7f882419c0a0 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8829058640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88241086a0 con 0x7f880c003530 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8822d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8822d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:37670/0 (socket says 172.21.15.194:37670) 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f8822d9d640 1 -- 172.21.15.194:0/128434542 learned_addr learned my addr 172.21.15.194:0/128434542 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f882259c640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 0x7f882419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.899+0000 7f882359e640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8822d9d640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 msgr2=0x7f8824197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T01:07:04.902 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8822d9d640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 msgr2=0x7f8824197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8822d9d640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8822d9d640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882359e640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 msgr2=0x7f882419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882359e640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 msgr2=0x7f882419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882359e640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882359e640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 msgr2=0x7f8824197290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 msgr2=0x7f882419c550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:04.903 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 -- 172.21.15.194:0/128434542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88241a3e10 con 0x7f88241977d0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f882259c640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 0x7f882419bb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f881800b450 tx=0x7f881800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818004230 con 0x7f88241977d0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88180043d0 con 0x7f88241977d0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88241a3fe0 con 0x7f88241977d0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8818003be0 con 0x7f88241977d0 2024-09-05T01:07:04.904 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.900+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a4540 con 0x7f88241977d0 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.901+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f881800f050 con 0x7f88241977d0 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.901+0000 7f880bfff640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 0x7f881003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.902+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f881804be40 con 0x7f88241977d0 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.902+0000 7f8822d9d640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 0x7f881003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.902+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88241095d0 con 0x7f88241977d0 2024-09-05T01:07:04.905 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.902+0000 7f8822d9d640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 0x7f881003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8814014040 tx=0x7f8814014930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:04.907 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:04.907+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f881800d040 con 0x7f88241977d0 2024-09-05T01:07:05.154 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.153+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f88241a47d0 con 0x7f88241977d0 2024-09-05T01:07:05.154 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: mon.c calling monitor election 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: mon.a calling monitor election 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.155 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: monmap epoch 3 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: min_mon_release 19 (squid) 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: election_strategy: 1 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: fsmap 2024-09-05T01:07:05.156 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: mgrmap e13: x(active, since 47s) 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: overall HEALTH_OK 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.157 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.157 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.154+0000 7f880bfff640 1 -- 172.21.15.194:0/128434542 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f8818021bf0 con 0x7f88241977d0 2024-09-05T01:07:05.157 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:07:05.158 INFO:teuthology.orchestra.run.smithi194.stdout:{"epoch":3,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","modified":"2024-09-05T01:06:59.735345Z","created":"2024-09-05T01:05:29.598613Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:3300","nonce":0},{"type":"v1","addr":"172.21.15.194:6789","nonce":0}]},"addr":"172.21.15.194:6789/0","public_addr":"172.21.15.194:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T01:07:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:dumped monmap epoch 3 2024-09-05T01:07:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 msgr2=0x7f881003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 0x7f881003b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8814014040 tx=0x7f8814014930 comp rx=0 tx=0).stop 2024-09-05T01:07:05.158 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 msgr2=0x7f882419bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 0x7f882419bb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f881800b450 tx=0x7f881800b910 comp rx=0 tx=0).stop 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 shutdown_connections 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88100394f0 0x7f881003b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f882419c0a0 0x7f882419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88241977d0 0x7f882419bb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 --2- 172.21.15.194:0/128434542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f880c003530 0x7f8824197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.156+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 >> 172.21.15.194:0/128434542 conn(0x7f88240fc220 msgr2=0x7f88241059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.157+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 shutdown_connections 2024-09-05T01:07:05.159 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:05.157+0000 7f8829058640 1 -- 172.21.15.194:0/128434542 wait complete. 2024-09-05T01:07:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:07:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: mon.c calling monitor election 2024-09-05T01:07:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: mon.a calling monitor election 2024-09-05T01:07:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:07:05.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: monmap epoch 3 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:05.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: election_strategy: 1 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:05.246 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: fsmap 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: mgrmap e13: x(active, since 47s) 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.247 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:05.696 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-05T01:07:05.696 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph config generate-minimal-conf 2024-09-05T01:07:05.957 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:07:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:05 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/128434542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:07:05.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:06.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:06.113 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:05 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/128434542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:07:06.114 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:06.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/1838671359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 msgr2=0x7fe5cc100d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/1838671359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc100d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009880 tx=0x7fe5c002a5d0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/1838671359 shutdown_connections 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/1838671359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc100d70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/1838671359 >> 172.21.15.129:0/1838671359 conn(0x7fe5cc076250 msgr2=0x7fe5cc076660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/1838671359 shutdown_connections 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/1838671359 wait complete. 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 Processor -- start 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.409+0000 7fe5d2bcb640 1 -- start start 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:06.435 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5cc198660 0x7fe5cc198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 0x7fe5cc1a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10a9f0 con 0x7fe5cc100990 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10a870 con 0x7fe5cc1a1190 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d2bcb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc10ab70 con 0x7fe5cc198660 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36122/0 (socket says 172.21.15.129:36122) 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 -- 172.21.15.129:0/631134227 learned_addr learned my addr 172.21.15.129:0/631134227 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5cbfff640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5cc198660 0x7fe5cc198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d1168640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 0x7fe5cc1a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 -- 172.21.15.129:0/631134227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5cc198660 msgr2=0x7fe5cc198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:06.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5cc198660 0x7fe5cc198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 -- 172.21.15.129:0/631134227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 msgr2=0x7fe5cc1a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 0x7fe5cc1a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d0967640 1 -- 172.21.15.129:0/631134227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc1a3d20 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.410+0000 7fe5d1168640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 0x7fe5cc1a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5d0967640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0031440 tx=0x7fe5c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c003f030 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc1a3fb0 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5c0003cb0 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0035070 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.411+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc1a4460 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.412+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe5c0042030 con 0x7fe5cc100990 2024-09-05T01:07:06.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.412+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc100df0 con 0x7fe5cc100990 2024-09-05T01:07:06.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.412+0000 7fe5c9ffb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 0x7fe5bc03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:06.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.412+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe5c006ce70 con 0x7fe5cc100990 2024-09-05T01:07:06.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.412+0000 7fe5cbfff640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 0x7fe5bc03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:06.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.413+0000 7fe5cbfff640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 0x7fe5bc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009a30 tx=0x7fe5b8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:06.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.417+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5c00438e0 con 0x7fe5cc100990 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.664+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe5cc02d7c0 con 0x7fe5cc100990 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.664+0000 7fe5c9ffb640 1 -- 172.21.15.129:0/631134227 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7fe5c00433c0 con 0x7fe5cc100990 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stdout:# minimal ceph.conf for c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stdout:[global] 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: fsid = c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:06.665 INFO:teuthology.orchestra.run.smithi129.stdout: mon_host = [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] 2024-09-05T01:07:06.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.666+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 msgr2=0x7fe5bc03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:06.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.666+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 0x7fe5bc03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe5b8009a30 tx=0x7fe5b8009190 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 msgr2=0x7fe5cc198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0031440 tx=0x7fe5c00042f0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 shutdown_connections 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5bc0394f0 0x7fe5bc03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe5cc1a1190 0x7fe5cc1a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe5cc198660 0x7fe5cc198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 --2- 172.21.15.129:0/631134227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5cc100990 0x7fe5cc198120 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 >> 172.21.15.129:0/631134227 conn(0x7fe5cc076250 msgr2=0x7fe5cc1020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 shutdown_connections 2024-09-05T01:07:06.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:06.667+0000 7fe5d2bcb640 1 -- 172.21.15.129:0/631134227 wait complete. 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: mon.c calling monitor election 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: mon.a calling monitor election 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: monmap epoch 3 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:07.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: min_mon_release 19 (squid) 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: election_strategy: 1 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: fsmap 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: mgrmap e13: x(active, since 47s) 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: overall HEALTH_OK 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:07.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/128434542' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T01:07:07.164 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:07.344 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-05T01:07:07.344 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:07:07.344 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:07:07.375 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:07:07.375 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:07:07.440 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:07:07.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:07:07.469 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:07:07.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:07:07.533 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:07:07.534 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T01:07:07.561 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:07:07.561 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T01:07:07.627 INFO:tasks.cephadm:Adding mgr.x on smithi129 2024-09-05T01:07:07.627 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-09-05T01:07:07.627 INFO:tasks.cephadm:Adding mgr.z on smithi194 2024-09-05T01:07:07.627 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch apply mgr '3;smithi129=x;smithi183=y;smithi194=z' 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mon.b calling monitor election 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi194:/etc/ceph/ceph.conf 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.628 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: Updating smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mon.b calling monitor election 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mon.c calling monitor election 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mon.a calling monitor election 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: monmap epoch 3 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: min_mon_release 19 (squid) 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: election_strategy: 1 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:07.629 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: fsmap 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: mgrmap e13: x(active, since 49s) 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.630 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:07.631 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mon.b calling monitor election 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi194:/etc/ceph/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: Updating smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mon.b calling monitor election 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mon.c calling monitor election 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mon.a calling monitor election 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: monmap epoch 3 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: min_mon_release 19 (squid) 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: election_strategy: 1 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: fsmap 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: mgrmap e13: x(active, since 49s) 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: overall HEALTH_OK 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:07.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:07.664 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mon.b calling monitor election 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi183:/etc/ceph/ceph.conf 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi194:/etc/ceph/ceph.conf 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.689 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: Updating smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mon.b calling monitor election 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mon.c calling monitor election 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mon.a calling monitor election 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: monmap epoch 3 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: last_changed 2024-09-05T01:06:59.735345+0000 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: created 2024-09-05T01:05:29.598613+0000 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: min_mon_release 19 (squid) 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: election_strategy: 1 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: 0: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.a 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: 1: [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] mon.c 2024-09-05T01:07:07.690 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: fsmap 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: mgrmap e13: x(active, since 49s) 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: overall HEALTH_OK 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.691 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.692 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:07.692 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:07.692 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:07.692 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:07.902 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:07:08.327 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 -- 172.21.15.194:0/2898143727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b41005f0 msgr2=0x7f98b41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 --2- 172.21.15.194:0/2898143727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b41005f0 0x7f98b41009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009880 tx=0x7f98a402a5d0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 -- 172.21.15.194:0/2898143727 shutdown_connections 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 --2- 172.21.15.194:0/2898143727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b41005f0 0x7f98b41009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 -- 172.21.15.194:0/2898143727 >> 172.21.15.194:0/2898143727 conn(0x7f98b40fbda0 msgr2=0x7f98b40fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 -- 172.21.15.194:0/2898143727 shutdown_connections 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.325+0000 7f98b8b70640 1 -- 172.21.15.194:0/2898143727 wait complete. 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 Processor -- start 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 -- start start 2024-09-05T01:07:08.328 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4197910 0x7f98b419ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4106590 con 0x7f98b4197490 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4106410 con 0x7f98b4197910 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b8b70640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f98b4106710 con 0x7f98b41005f0 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98abfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98abfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.194:58074/0 (socket says 172.21.15.194:58074) 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:55058/0 (socket says 172.21.15.194:55058) 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98abfff640 1 -- 172.21.15.194:0/1429077816 learned_addr learned my addr 172.21.15.194:0/1429077816 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b2d9d640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4197910 0x7f98b419ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 -- 172.21.15.194:0/1429077816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4197910 msgr2=0x7f98b419ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:08.329 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4197910 0x7f98b419ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 -- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 msgr2=0x7f98b419c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98b259c640 1 -- 172.21.15.194:0/1429077816 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b419f350 con 0x7f98b41005f0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.326+0000 7f98abfff640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98b259c640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009880 tx=0x7f98a40042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a403f030 con 0x7f98b41005f0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98a40044d0 con 0x7f98b41005f0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b419f5e0 con 0x7f98b41005f0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4035070 con 0x7f98b41005f0 2024-09-05T01:07:08.330 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.327+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b419fb40 con 0x7f98b41005f0 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f98a4042030 con 0x7f98b41005f0 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98ab7fe640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 0x7f98a003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98a406dc40 con 0x7f98b41005f0 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b4100a70 con 0x7f98b41005f0 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98abfff640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 0x7f98a003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:08.331 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.328+0000 7f98abfff640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 0x7f98a003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98b4198470 tx=0x7f989c009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:08.333 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.333+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98a4036260 con 0x7f98b41005f0 2024-09-05T01:07:08.511 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.510+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}) -- 0x7f98b4062630 con 0x7f98a00394f0 2024-09-05T01:07:08.517 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.516+0000 7f98ab7fe640 1 -- 172.21.15.194:0/1429077816 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f98b4062630 con 0x7f98a00394f0 2024-09-05T01:07:08.517 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mgr update... 2024-09-05T01:07:08.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 msgr2=0x7f98a003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:08.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 0x7f98a003b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f98b4198470 tx=0x7f989c009240 comp rx=0 tx=0).stop 2024-09-05T01:07:08.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 msgr2=0x7f98b4196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:08.519 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98a4009880 tx=0x7f98a40042e0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 shutdown_connections 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98a00394f0 0x7f98a003b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b4197910 0x7f98b419ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98b4197490 0x7f98b419c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 --2- 172.21.15.194:0/1429077816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f98b41005f0 0x7f98b4196f50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.518+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 >> 172.21.15.194:0/1429077816 conn(0x7f98b40fbda0 msgr2=0x7f98b406ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.519+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 shutdown_connections 2024-09-05T01:07:08.520 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:07:08.519+0000 7f98b8b70640 1 -- 172.21.15.194:0/1429077816 wait complete. 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: Reconfiguring daemon mon.a on smithi129 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:08.937 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: Reconfiguring daemon mon.a on smithi129 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:08.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: Reconfiguring daemon mon.a on smithi129 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:09.100 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y.service 2024-09-05T01:07:09.102 DEBUG:teuthology.orchestra.run.smithi194:mgr.z> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.z.service 2024-09-05T01:07:09.142 INFO:tasks.cephadm:Deploying OSDs... 2024-09-05T01:07:09.142 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:07:09.142 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:07:09.159 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:07:09.159 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-05 01:06:28.446308296 +0000 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-05 01:02:19.186759774 +0000 2024-09-05T01:07:09.215 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-05 01:02:19.186759774 +0000 2024-09-05T01:07:09.216 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-05 01:02:19.186759774 +0000 2024-09-05T01:07:09.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:07:09.279 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-05T01:07:09.279 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-05T01:07:09.280 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000212606 s, 2.4 MB/s 2024-09-05T01:07:09.281 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:07:09.337 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-05T01:07:09.392 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:07:09.392 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-05 01:06:28.479307635 +0000 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-05 01:02:19.524752859 +0000 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-05 01:02:19.524752859 +0000 2024-09-05T01:07:09.393 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-05 01:02:19.524752859 +0000 2024-09-05T01:07:09.393 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:07:09.455 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-05T01:07:09.455 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-05T01:07:09.456 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000195246 s, 2.6 MB/s 2024-09-05T01:07:09.457 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:07:09.513 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-05T01:07:09.568 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:07:09.568 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-05 01:06:28.508307054 +0000 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-05 01:02:19.848746230 +0000 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-05 01:02:19.848746230 +0000 2024-09-05T01:07:09.569 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-05 01:02:19.848746230 +0000 2024-09-05T01:07:09.569 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:07:09.632 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-05T01:07:09.632 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-05T01:07:09.632 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000310913 s, 1.6 MB/s 2024-09-05T01:07:09.634 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:07:09.689 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-05T01:07:09.745 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-05 01:06:28.535306514 +0000 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-05 01:02:20.170739642 +0000 2024-09-05T01:07:09.746 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-05 01:02:20.170739642 +0000 2024-09-05T01:07:09.747 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-05 01:02:20.170739642 +0000 2024-09-05T01:07:09.747 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:07:09.809 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-05T01:07:09.809 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-05T01:07:09.810 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.0002152 s, 2.4 MB/s 2024-09-05T01:07:09.811 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:07:09.869 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-09-05T01:07:09.869 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:07:09.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:07:09.892 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-09-05T01:07:09.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:09 smithi129 ceph-mon[30979]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:09.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:09 smithi129 ceph-mon[30979]: Saving service mgr spec with placement smithi129=x;smithi183=y;smithi194=z;count:3 2024-09-05T01:07:09.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:09 smithi129 ceph-mon[30979]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:07:09.938 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:09 smithi129 ceph-mon[30979]: Reconfiguring daemon mgr.x on smithi129 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-05 01:06:59.177382273 +0000 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-05 01:02:19.626702970 +0000 2024-09-05T01:07:09.954 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-05 01:02:19.626702970 +0000 2024-09-05T01:07:09.955 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-05 01:02:19.626702970 +0000 2024-09-05T01:07:09.955 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:07:10.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:09 smithi194 ceph-mon[34941]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:10.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:09 smithi194 ceph-mon[34941]: Saving service mgr spec with placement smithi129=x;smithi183=y;smithi194=z;count:3 2024-09-05T01:07:10.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:09 smithi194 ceph-mon[34941]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:07:10.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:09 smithi194 ceph-mon[34941]: Reconfiguring daemon mgr.x on smithi129 2024-09-05T01:07:10.019 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:09 smithi183 ceph-mon[34589]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi129=x;smithi183=y;smithi194=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:10.019 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:09 smithi183 ceph-mon[34589]: Saving service mgr spec with placement smithi129=x;smithi183=y;smithi194=z;count:3 2024-09-05T01:07:10.019 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:09 smithi183 ceph-mon[34589]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T01:07:10.019 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:09 smithi183 ceph-mon[34589]: Reconfiguring daemon mgr.x on smithi129 2024-09-05T01:07:10.022 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-05T01:07:10.022 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-05T01:07:10.022 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000293845 s, 1.7 MB/s 2024-09-05T01:07:10.023 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:07:10.080 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-09-05T01:07:10.137 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-05 01:06:59.178382263 +0000 2024-09-05T01:07:10.138 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-05 01:02:19.946699702 +0000 2024-09-05T01:07:10.139 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-05 01:02:19.946699702 +0000 2024-09-05T01:07:10.139 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-05 01:02:19.946699702 +0000 2024-09-05T01:07:10.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:07:10.201 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-05T01:07:10.201 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-05T01:07:10.201 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000186054 s, 2.8 MB/s 2024-09-05T01:07:10.202 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:07:10.258 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-05 01:06:59.178382263 +0000 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-05 01:02:20.265696447 +0000 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-05 01:02:20.265696447 +0000 2024-09-05T01:07:10.314 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-05 01:02:20.265696447 +0000 2024-09-05T01:07:10.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:07:10.378 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-05T01:07:10.378 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-05T01:07:10.378 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000173406 s, 3.0 MB/s 2024-09-05T01:07:10.379 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:07:10.436 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-09-05T01:07:10.491 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:07:10.491 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:10.491 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T01:07:10.491 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:10.492 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:10.492 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-09-05 01:06:59.179382252 +0000 2024-09-05T01:07:10.492 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-09-05 01:02:20.613692897 +0000 2024-09-05T01:07:10.492 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-09-05 01:02:20.613692897 +0000 2024-09-05T01:07:10.492 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-09-05 01:02:20.613692897 +0000 2024-09-05T01:07:10.492 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:07:10.556 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-09-05T01:07:10.556 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-09-05T01:07:10.556 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000183729 s, 2.8 MB/s 2024-09-05T01:07:10.557 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:07:10.614 DEBUG:teuthology.orchestra.run.smithi194:> set -ex 2024-09-05T01:07:10.614 DEBUG:teuthology.orchestra.run.smithi194:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T01:07:10.634 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T01:07:10.634 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_1 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-05 01:06:49.955951757 +0000 2024-09-05T01:07:10.693 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-05 01:02:19.330325111 +0000 2024-09-05T01:07:10.694 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-05 01:02:19.330325111 +0000 2024-09-05T01:07:10.694 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-05 01:02:19.330325111 +0000 2024-09-05T01:07:10.694 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T01:07:10.763 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-05T01:07:10.763 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-05T01:07:10.763 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000291568 s, 1.8 MB/s 2024-09-05T01:07:10.764 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T01:07:10.823 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_2 2024-09-05T01:07:10.880 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T01:07:10.880 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:10.880 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-05T01:07:10.880 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:10.880 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:10.881 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-05 01:06:49.955951757 +0000 2024-09-05T01:07:10.881 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-05 01:02:19.668318071 +0000 2024-09-05T01:07:10.881 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-05 01:02:19.668318071 +0000 2024-09-05T01:07:10.881 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-05 01:02:19.668318071 +0000 2024-09-05T01:07:10.881 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T01:07:10.946 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-05T01:07:10.946 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-05T01:07:10.946 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000232354 s, 2.2 MB/s 2024-09-05T01:07:10.947 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:11.005 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_3 2024-09-05T01:07:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T01:07:11.061 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-05 01:06:49.956951736 +0000 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-05 01:02:20.002311115 +0000 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-05 01:02:20.002311115 +0000 2024-09-05T01:07:11.062 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-05 01:02:20.002311115 +0000 2024-09-05T01:07:11.062 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T01:07:11.126 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-05T01:07:11.126 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-05T01:07:11.126 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000210841 s, 2.4 MB/s 2024-09-05T01:07:11.127 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:11.184 DEBUG:teuthology.orchestra.run.smithi194:> stat /dev/vg_nvme/lv_4 2024-09-05T01:07:11.239 INFO:teuthology.orchestra.run.smithi194.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T01:07:11.239 INFO:teuthology.orchestra.run.smithi194.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Access: 2024-09-05 01:06:49.956951736 +0000 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Modify: 2024-09-05 01:02:20.325304388 +0000 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout:Change: 2024-09-05 01:02:20.325304388 +0000 2024-09-05T01:07:11.240 INFO:teuthology.orchestra.run.smithi194.stdout: Birth: 2024-09-05 01:02:20.325304388 +0000 2024-09-05T01:07:11.240 DEBUG:teuthology.orchestra.run.smithi194:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T01:07:11.302 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records in 2024-09-05T01:07:11.302 INFO:teuthology.orchestra.run.smithi194.stderr:1+0 records out 2024-09-05T01:07:11.302 INFO:teuthology.orchestra.run.smithi194.stderr:512 bytes copied, 0.000206493 s, 2.5 MB/s 2024-09-05T01:07:11.304 DEBUG:teuthology.orchestra.run.smithi194:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T01:07:11.360 INFO:tasks.cephadm:Deploying osd.0 on smithi129 with /dev/vg_nvme/lv_4... 2024-09-05T01:07:11.361 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:07:11.615 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:07:11.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:07:11.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: Reconfiguring daemon mon.b on smithi183 2024-09-05T01:07:11.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:11.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:11.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:11.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:12.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: Reconfiguring daemon mon.b on smithi183 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:12.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:12.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: Reconfiguring mon.b (monmap changed)... 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: Reconfiguring daemon mon.b on smithi183 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T01:07:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:12.849 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:12 smithi129 ceph-mon[30979]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:07:12.849 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:12 smithi129 ceph-mon[30979]: Reconfiguring daemon mon.c on smithi194 2024-09-05T01:07:12.849 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:12 smithi129 ceph-mon[30979]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:13.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:12 smithi194 ceph-mon[34941]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:07:13.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:12 smithi194 ceph-mon[34941]: Reconfiguring daemon mon.c on smithi194 2024-09-05T01:07:13.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:12 smithi194 ceph-mon[34941]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:12 smithi183 ceph-mon[34589]: Reconfiguring mon.c (monmap changed)... 2024-09-05T01:07:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:12 smithi183 ceph-mon[34589]: Reconfiguring daemon mon.c on smithi194 2024-09-05T01:07:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:12 smithi183 ceph-mon[34589]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:13.976 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:13.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:13.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:13.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:13.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:13 smithi129 ceph-mon[30979]: Deploying daemon mgr.z on smithi194 2024-09-05T01:07:14.003 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:14.116 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:14.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:14.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:14.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:14.117 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:13 smithi194 ceph-mon[34941]: Deploying daemon mgr.z on smithi194 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:13 smithi183 ceph-mon[34589]: Deploying daemon mgr.z on smithi194 2024-09-05T01:07:14.245 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:07:14.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 -- 172.21.15.129:0/1475303492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2190101cf0 msgr2=0x7f2190102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 --2- 172.21.15.129:0/1475303492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2190101cf0 0x7f2190102170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2178009900 tx=0x7f217802a410 comp rx=0 tx=0).stop 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 -- 172.21.15.129:0/1475303492 shutdown_connections 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 --2- 172.21.15.129:0/1475303492 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f21901026b0 0x7f2190108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 --2- 172.21.15.129:0/1475303492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2190101cf0 0x7f2190102170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 --2- 172.21.15.129:0/1475303492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f2190100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 -- 172.21.15.129:0/1475303492 >> 172.21.15.129:0/1475303492 conn(0x7f21900fc2a0 msgr2=0x7f21900fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 -- 172.21.15.129:0/1475303492 shutdown_connections 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.658+0000 7f2195594640 1 -- 172.21.15.129:0/1475303492 wait complete. 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 Processor -- start 2024-09-05T01:07:14.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 -- start start 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f219006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2190101cf0 0x7f219019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f219010c6d0 con 0x7f21901026b0 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f219010c550 con 0x7f2190100af0 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.659+0000 7f2195594640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f219010c850 con 0x7f2190101cf0 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33508/0 (socket says 172.21.15.129:33508) 2024-09-05T01:07:14.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 -- 172.21.15.129:0/2306478602 learned_addr learned my addr 172.21.15.129:0/2306478602 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f2187fff640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2190101cf0 0x7f219019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218effd640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f219006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 -- 172.21.15.129:0/2306478602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2190101cf0 msgr2=0x7f219019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2190101cf0 0x7f219019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 -- 172.21.15.129:0/2306478602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 msgr2=0x7f219006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f219006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 -- 172.21.15.129:0/2306478602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219019fb20 con 0x7f21901026b0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218effd640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f219006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218f7fe640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f218800b970 tx=0x7f218800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188019070 con 0x7f21901026b0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2188014440 con 0x7f21901026b0 2024-09-05T01:07:14.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219019fdb0 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880075d0 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.660+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21901a0340 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.661+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f218801c070 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.661+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2190107f90 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.662+0000 7f218cff9640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 0x7f217c03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.662+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f218804bab0 con 0x7f21901026b0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.662+0000 7f218effd640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 0x7f217c03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:14.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.662+0000 7f218effd640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 0x7f217c03b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21800058c0 tx=0x7f2180005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:14.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.666+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2188012500 con 0x7f21901026b0 2024-09-05T01:07:14.740 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:14 smithi194 systemd[1]: Starting Ceph mgr.z for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:07:14.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:14.844+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2190062630 con 0x7f217c039290 2024-09-05T01:07:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:14 smithi129 ceph-mon[30979]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:15.186 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:14 smithi183 ceph-mon[34589]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:15.187 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:14 smithi194 ceph-mon[34941]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:15.187 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:14 smithi194 podman[36740]: 2024-09-05 01:07:14.642104882 +0000 UTC m=+0.016267588 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:07:15.187 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:14 smithi194 podman[36740]: 2024-09-05 01:07:14.770604677 +0000 UTC m=+0.144767373 container create 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e) 2024-09-05T01:07:15.187 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:14 smithi194 podman[36740]: 2024-09-05 01:07:14.99534249 +0000 UTC m=+0.369505183 container init 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T01:07:15.253 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:14 smithi194 podman[36740]: 2024-09-05 01:07:14.999150098 +0000 UTC m=+0.373312793 container start 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T01:07:15.254 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mgr[36759]: -- 172.21.15.194:0/2514890647 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a256c330e0 con 0x55a256c10800 2024-09-05T01:07:15.254 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 bash[36740]: 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac 2024-09-05T01:07:15.254 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 systemd[1]: Started Ceph mgr.z for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:07:15.254 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.126+0000 7f437607f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:07:15.255 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.185+0000 7f437607f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:07:15.684 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.333+0000 7f437607f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:07:15.684 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.399+0000 7f437607f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:07:15.685 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.546+0000 7f437607f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:07:15.685 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.683+0000 7f437607f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:15.883 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:15.884 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:15.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:15.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:15.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:15.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.996 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:15.997 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T01:07:15.997 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T01:07:15.997 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T01:07:15.997 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:16.250 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:15.995+0000 7f437607f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:07:16.250 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:16.055+0000 7f437607f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:07:16.250 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:16.244+0000 7f437607f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:07:16.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:16 smithi129 ceph-mon[30979]: Deploying daemon mgr.y on smithi183 2024-09-05T01:07:16.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:16 smithi129 ceph-mon[30979]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:17.159 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:16 smithi183 ceph-mon[34589]: Deploying daemon mgr.y on smithi183 2024-09-05T01:07:17.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:16 smithi183 ceph-mon[34589]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:17.160 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:16 smithi183 systemd[1]: Starting Ceph mgr.y for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:07:17.163 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:16 smithi194 ceph-mon[34941]: Deploying daemon mgr.y on smithi183 2024-09-05T01:07:17.163 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:16 smithi194 ceph-mon[34941]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:17.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 podman[35933]: 2024-09-05 01:07:17.061467867 +0000 UTC m=+0.027561014 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:07:17.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 podman[35933]: 2024-09-05 01:07:17.162605252 +0000 UTC m=+0.128698429 container create a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-05T01:07:17.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 podman[35933]: 2024-09-05 01:07:17.381378169 +0000 UTC m=+0.347471303 container init a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-05T01:07:17.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 podman[35933]: 2024-09-05 01:07:17.384176616 +0000 UTC m=+0.350269756 container start a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-05T01:07:17.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mgr[35952]: -- 172.21.15.183:0/1832752092 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c495fbf0e0 con 0x55c495f9d400 2024-09-05T01:07:17.500 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:17.162+0000 7f437607f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:07:17.500 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:17.229+0000 7f437607f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:07:17.722 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 bash[35933]: a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e 2024-09-05T01:07:17.722 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 systemd[1]: Started Ceph mgr.y for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:07:17.722 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:17.505+0000 7f36d6726040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T01:07:17.722 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:17.565+0000 7f36d6726040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T01:07:17.722 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:17.721+0000 7f36d6726040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:17.791+0000 7f36d6726040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:17.955+0000 7f36d6726040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:18.105+0000 7f36d6726040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]: dispatch 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]': finished 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:07:18.106 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2846458715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:18.107 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:17.997+0000 7f437607f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.119+0000 7f437607f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]: dispatch 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]': finished 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:07:18.120 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2846458715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:18.121 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]: dispatch 2024-09-05T01:07:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1129571737' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8916e3b-e0c4-45b9-84bb-5f527d0f0439"}]': finished 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2846458715' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:18.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:18.452 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.260+0000 7f437607f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:07:18.452 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.324+0000 7f437607f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:07:18.682 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:18.429+0000 7f36d6726040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T01:07:18.682 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:18.491+0000 7f36d6726040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T01:07:18.750 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.481+0000 7f437607f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:07:18.750 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.550+0000 7f437607f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:07:18.750 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.610+0000 7f437607f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:07:19.057 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-mon[34941]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:19.058 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.058 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.058 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.775+0000 7f437607f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:07:19.058 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.842+0000 7f437607f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:07:19.059 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.912+0000 7f437607f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:07:19.059 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:18.995+0000 7f437607f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:07:19.143 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:18 smithi129 ceph-mon[30979]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:19.143 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.143 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.161 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:18.681+0000 7f36d6726040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T01:07:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-mon[34589]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.500 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:19.056+0000 7f437607f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:07:19.500 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:19.115+0000 7f437607f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:07:19.911 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:19.573+0000 7f36d6726040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T01:07:19.911 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:19.640+0000 7f36d6726040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:19.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:19.913 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:19.913 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:19.913 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.250 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z[36755]: 2024-09-05T01:07:19.901+0000 7f437607f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:07:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:20.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:20.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:20.602 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.348+0000 7f36d6726040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T01:07:20.602 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.461+0000 7f36d6726040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T01:07:20.602 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.601+0000 7f36d6726040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T01:07:20.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:20.804+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f2188007850 con 0x7f21901026b0 2024-09-05T01:07:20.878 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.662+0000 7f36d6726040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.810+0000 7f36d6726040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.877+0000 7f36d6726040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: Standby manager daemon z started 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:07:20.879 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:21.156 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:20.937+0000 7f36d6726040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T01:07:21.156 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.091+0000 7f36d6726040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T01:07:21.156 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.155+0000 7f36d6726040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: Standby manager daemon z started 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:07:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:20 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: Standby manager daemon z started 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T01:07:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:20 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.194:0/644356838' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:21.410 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.221+0000 7f36d6726040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T01:07:21.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.301+0000 7f36d6726040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T01:07:21.411 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.359+0000 7f36d6726040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T01:07:21.810 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:21.417+0000 7f36d6726040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T01:07:22.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-mon[34589]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-05T01:07:22.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:07:22.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:21 smithi129 ceph-mon[30979]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-05T01:07:22.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:07:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:21 smithi194 ceph-mon[34941]: mgrmap e14: x(active, since 63s), standbys: z 2024-09-05T01:07:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T01:07:22.661 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y[35948]: 2024-09-05T01:07:22.201+0000 7f36d6726040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T01:07:22.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:22.819+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mon.0 v2:172.21.15.129:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f2188007bd0 con 0x7f21901026b0 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: Standby manager daemon y started 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:07:23.138 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:22 smithi129 ceph-mon[30979]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: Standby manager daemon y started 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:07:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:22 smithi183 ceph-mon[34589]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: Standby manager daemon y started 2024-09-05T01:07:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T01:07:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T01:07:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T01:07:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:22 smithi194 ceph-mon[34941]: from='mgr.? 172.21.15.183:0/335892205' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T01:07:23.854 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:23 smithi129 ceph-mon[30979]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-05T01:07:23.854 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:07:24.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:23 smithi183 ceph-mon[34589]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-05T01:07:24.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:07:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:23 smithi194 ceph-mon[34941]: mgrmap e15: x(active, since 65s), standbys: z, y 2024-09-05T01:07:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T01:07:25.001 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:24 smithi129 ceph-mon[30979]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:25.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:24 smithi183 ceph-mon[34589]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:24 smithi194 ceph-mon[34941]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:25.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:07:25.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:26.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:07:26.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:26.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T01:07:26.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:26.893 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:26 smithi129 ceph-mon[30979]: Deploying daemon osd.0 on smithi129 2024-09-05T01:07:26.893 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:26 smithi129 ceph-mon[30979]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:26 smithi183 ceph-mon[34589]: Deploying daemon osd.0 on smithi129 2024-09-05T01:07:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:26 smithi183 ceph-mon[34589]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:26 smithi194 ceph-mon[34941]: Deploying daemon osd.0 on smithi129 2024-09-05T01:07:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:26 smithi194 ceph-mon[34941]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:28 smithi183 ceph-mon[34589]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:28 smithi129 ceph-mon[30979]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:28 smithi194 ceph-mon[34941]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:30 smithi183 ceph-mon[34589]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:30 smithi129 ceph-mon[30979]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:30 smithi194 ceph-mon[34941]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:33.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:33.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:33.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:34.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.463+0000 7f218cff9640 1 -- 172.21.15.129:0/2306478602 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2190062630 con 0x7f217c039290 2024-09-05T01:07:34.465 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 0 on host 'smithi129' 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 msgr2=0x7f217c03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 0x7f217c03b750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21800058c0 tx=0x7f2180005850 comp rx=0 tx=0).stop 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 msgr2=0x7f219019f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f218800b970 tx=0x7f218800be30 comp rx=0 tx=0).stop 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 shutdown_connections 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f217c039290 0x7f217c03b750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f21901026b0 0x7f219019f210 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2190101cf0 0x7f219019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:34.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 --2- 172.21.15.129:0/2306478602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2190100af0 0x7f219006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:34.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 >> 172.21.15.129:0/2306478602 conn(0x7f21900fc2a0 msgr2=0x7f2190105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:34.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 shutdown_connections 2024-09-05T01:07:34.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:07:34.465+0000 7f2195594640 1 -- 172.21.15.129:0/2306478602 wait complete. 2024-09-05T01:07:35.120 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:34 smithi129 ceph-mon[30979]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:35.120 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:35.120 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.120 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.150 DEBUG:teuthology.orchestra.run.smithi129:osd.0> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.0.service 2024-09-05T01:07:35.153 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-09-05T01:07:35.153 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:07:35.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:34 smithi183 ceph-mon[34589]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:35.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:34 smithi194 ceph-mon[34941]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:35.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:35.404 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:07:36.492 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:07:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:07:36.186+0000 7f786d43d640 -1 osd.0 0 log_to_monitors true 2024-09-05T01:07:37.092 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:36 smithi183 ceph-mon[34589]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:37.092 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:36 smithi183 ceph-mon[34589]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:07:37.172 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:36 smithi129 ceph-mon[30979]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:37.173 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:36 smithi129 ceph-mon[30979]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:07:37.185 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:36 smithi194 ceph-mon[34941]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:37.185 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:36 smithi194 ceph-mon[34941]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T01:07:37.774 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:07:37.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-09-05T01:07:38.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:37 smithi194 ceph-mon[34941]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:07:38.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:37 smithi194 ceph-mon[34941]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:07:38.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:37 smithi194 ceph-mon[34941]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:07:38.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:37 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:38.079 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:07:38.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:37 smithi183 ceph-mon[34589]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:07:38.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:37 smithi183 ceph-mon[34589]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:07:38.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:37 smithi183 ceph-mon[34589]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:07:38.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:37 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-mon[30979]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T01:07:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-mon[30979]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T01:07:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-mon[30979]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:07:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:38.243 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:07:37.863+0000 7f78693c0640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T01:07:38.243 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:07:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:07:37.866+0000 7f78651e8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:07:38.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 -- 172.21.15.183:0/1139282613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4100ad0 msgr2=0x7f76c4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:38.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/1139282613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4100ad0 0x7f76c4100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402a410 comp rx=0 tx=0).stop 2024-09-05T01:07:38.549 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 -- 172.21.15.183:0/1139282613 shutdown_connections 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/1139282613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/1139282613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4101cd0 0x7f76c4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/1139282613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4100ad0 0x7f76c4100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 -- 172.21.15.183:0/1139282613 >> 172.21.15.183:0/1139282613 conn(0x7f76c40fc280 msgr2=0x7f76c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 -- 172.21.15.183:0/1139282613 shutdown_connections 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.531+0000 7f76c9c2d640 1 -- 172.21.15.183:0/1139282613 wait complete. 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 Processor -- start 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 -- start start 2024-09-05T01:07:38.550 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4100ad0 0x7f76c4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c419f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76c410d7b0 con 0x7f76c4100ad0 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76c410d630 con 0x7f76c4101cd0 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c9c2d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f76c410d930 con 0x7f76c4102690 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:54102/0 (socket says 172.21.15.183:54102) 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 -- 172.21.15.183:0/3292059190 learned_addr learned my addr 172.21.15.183:0/3292059190 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c3fff640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c419f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:38.551 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 -- 172.21.15.183:0/3292059190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 msgr2=0x7f76c419f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c419f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 -- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4100ad0 msgr2=0x7f76c4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c37fe640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4100ad0 0x7f76c4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4100ad0 0x7f76c4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 -- 172.21.15.183:0/3292059190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c419fa60 con 0x7f76c4101cd0 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76c3fff640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.532+0000 7f76bbfff640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b4a0 tx=0x7f76ac00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.533+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac014070 con 0x7f76c4101cd0 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.533+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c419fd50 con 0x7f76c4101cd0 2024-09-05T01:07:38.552 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.533+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76ac010ac0 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.533+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac017c30 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.533+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a02b0 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.534+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c4107f70 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.535+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f76ac017070 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.535+0000 7f76c17fa640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 0x7f76b00a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.536+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f76ac0ccc70 con 0x7f76c4101cd0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.536+0000 7f76c37fe640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 0x7f76b00a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.536+0000 7f76c37fe640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 0x7f76b00a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76b402a930 tx=0x7f76b4005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:07:38.553 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.539+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76ac09b760 con 0x7f76c4101cd0 2024-09-05T01:07:38.716 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:38.714+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f76c4062630 con 0x7f76b00a2b20 2024-09-05T01:07:39.116 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi129 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:39.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:07:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:07:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi129 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:39.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T01:07:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:07:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi129 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:39.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:39.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:07:39.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:07:39.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:07:39.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] boot 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:40.145 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:39 smithi183 ceph-mon[34589]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] boot 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:39 smithi129 ceph-mon[30979]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: osd.0 [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] boot 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T01:07:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:39 smithi194 ceph-mon[34941]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:40 smithi183 ceph-mon[34589]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:07:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:40 smithi129 ceph-mon[30979]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:07:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:40 smithi194 ceph-mon[34941]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/1457495675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]': finished 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:07:41.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/1791852689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:41.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:41 smithi183 ceph-mon[34589]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/1457495675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]': finished 2024-09-05T01:07:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:07:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:07:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/1791852689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:41 smithi129 ceph-mon[30979]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/1457495675' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]: dispatch 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b6b99a10-04d7-40ff-8801-2acf7628a478"}]': finished 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/1791852689' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:07:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:41 smithi194 ceph-mon[34941]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:44.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:44 smithi129 ceph-mon[30979]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:45.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:44 smithi194 ceph-mon[34941]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:44 smithi183 ceph-mon[34589]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:46.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:46 smithi183 ceph-mon[34589]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:46 smithi129 ceph-mon[30979]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:47.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:46 smithi194 ceph-mon[34941]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:48.858 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:48 smithi183 ceph-mon[34589]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:48.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:48 smithi129 ceph-mon[30979]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:49.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:48 smithi194 ceph-mon[34941]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:49.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:49 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:07:49.897 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:49 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:50.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:49 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:07:50.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:49 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:49 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T01:07:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:49 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:50.801 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:50 smithi183 ceph-mon[34589]: Deploying daemon osd.1 on smithi183 2024-09-05T01:07:50.802 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:50 smithi183 ceph-mon[34589]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:51.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:50 smithi194 ceph-mon[34941]: Deploying daemon osd.1 on smithi183 2024-09-05T01:07:51.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:50 smithi194 ceph-mon[34941]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:51.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:50 smithi129 ceph-mon[30979]: Deploying daemon osd.1 on smithi183 2024-09-05T01:07:51.161 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:50 smithi129 ceph-mon[30979]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:52.781 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:52 smithi183 ceph-mon[34589]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:53.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:52 smithi194 ceph-mon[34941]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:52 smithi129 ceph-mon[30979]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:55.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:54 smithi183 ceph-mon[34589]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:55.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:55.078 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:55.079 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:54 smithi129 ceph-mon[30979]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:54 smithi194 ceph-mon[34941]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:55.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:55.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:55.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:56.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:56 smithi183 ceph-mon[34589]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:57.052 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:56 smithi194 ceph-mon[34941]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:56 smithi129 ceph-mon[30979]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:57.989 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.988+0000 7f76c17fa640 1 -- 172.21.15.183:0/3292059190 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76c4062630 con 0x7f76b00a2b20 2024-09-05T01:07:57.990 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-09-05T01:07:57.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 msgr2=0x7f76b00a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:57.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 0x7f76b00a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f76b402a930 tx=0x7f76b4005fd0 comp rx=0 tx=0).stop 2024-09-05T01:07:57.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 msgr2=0x7f76c4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:07:57.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00b4a0 tx=0x7f76ac00b960 comp rx=0 tx=0).stop 2024-09-05T01:07:57.991 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 shutdown_connections 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f76b00a2b20 0x7f76b00a4fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f76c4102690 0x7f76c419f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f76c4101cd0 0x7f76c4198370 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 --2- 172.21.15.183:0/3292059190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76c4100ad0 0x7f76c4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 >> 172.21.15.183:0/3292059190 conn(0x7f76c40fc280 msgr2=0x7f76c4105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 shutdown_connections 2024-09-05T01:07:57.992 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:07:57.990+0000 7f76c9c2d640 1 -- 172.21.15.183:0/3292059190 wait complete. 2024-09-05T01:07:58.320 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.321 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:07:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:07:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:07:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:07:58.648 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.1.service 2024-09-05T01:07:58.651 INFO:tasks.cephadm:Deploying osd.2 on smithi194 with /dev/vg_nvme/lv_4... 2024-09-05T01:07:58.651 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T01:07:58.904 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:08:00.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:00 smithi129 ceph-mon[30979]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:01.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:00 smithi194 ceph-mon[34941]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:00 smithi183 ceph-mon[34589]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:01.161 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:08:00.745+0000 7fa112afa640 -1 osd.1 0 log_to_monitors true 2024-09-05T01:08:01.369 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:08:01.397 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_4 2024-09-05T01:08:01.654 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:01.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:01 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:01.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:01.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:01.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:01.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:01.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:01.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:01 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:02.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:02.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:01 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:02.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 -- 172.21.15.194:0/2739556151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6480100b10 msgr2=0x7f6480100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:02.111 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 --2- 172.21.15.194:0/2739556151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6480100b10 0x7f6480100f10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6474009900 tx=0x7f647402a410 comp rx=0 tx=0).stop 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 -- 172.21.15.194:0/2739556151 shutdown_connections 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 --2- 172.21.15.194:0/2739556151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f64801026d0 0x7f6480108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 --2- 172.21.15.194:0/2739556151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 0x7f6480102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 --2- 172.21.15.194:0/2739556151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6480100b10 0x7f6480100f10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 -- 172.21.15.194:0/2739556151 >> 172.21.15.194:0/2739556151 conn(0x7f64800fc2a0 msgr2=0x7f64800fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 -- 172.21.15.194:0/2739556151 shutdown_connections 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.108+0000 7f6485517640 1 -- 172.21.15.194:0/2739556151 wait complete. 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 Processor -- start 2024-09-05T01:08:02.112 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 -- start start 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6480100b10 0x7f648006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 0x7f648019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f648010c640 con 0x7f6480100b10 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f648010c4c0 con 0x7f6480101d10 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.109+0000 7f6485517640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f648010c7c0 con 0x7f64801026d0 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:35374/0 (socket says 172.21.15.194:35374) 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 -- 172.21.15.194:0/1318787255 learned_addr learned my addr 172.21.15.194:0/1318787255 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:08:02.113 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647e7fc640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 0x7f648019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647effd640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6480100b10 0x7f648006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 -- 172.21.15.194:0/1318787255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 msgr2=0x7f648019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 0x7f648019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 -- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6480100b10 msgr2=0x7f648006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6480100b10 0x7f648006b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 -- 172.21.15.194:0/1318787255 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648019faf0 con 0x7f64801026d0 2024-09-05T01:08:02.114 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f647f7fe640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f647800eaa0 tx=0x7f647800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.110+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647800cdb0 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.111+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6478005020 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.111+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648019fd80 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.111+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780116d0 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.111+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64801a02c0 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.112+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6480108050 con 0x7f64801026d0 2024-09-05T01:08:02.115 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.114+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f64780051d0 con 0x7f64801026d0 2024-09-05T01:08:02.116 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.115+0000 7f6467fff640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 0x7f64680a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:02.116 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.115+0000 7f647effd640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 0x7f64680a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:02.116 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.115+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7f64780cdb20 con 0x7f64801026d0 2024-09-05T01:08:02.116 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.115+0000 7f647effd640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 0x7f64680a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f64740314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:02.118 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.118+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6478016030 con 0x7f64801026d0 2024-09-05T01:08:02.296 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:02.296+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6480062630 con 0x7f64680a2b70 2024-09-05T01:08:02.984 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi183 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:02.985 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:03.161 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:08:02.751+0000 7fa10ea7d640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T01:08:03.161 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:08:02.755+0000 7fa10a8a5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:08:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi183 2024-09-05T01:08:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-05T01:08:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:08:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi183 2024-09-05T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 17088M 2024-09-05T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:03.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:08:03.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:03.982 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:03 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:08:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:08:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:08:04.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:08:04.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:04.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:03 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:08:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:08:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:08:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T01:08:04.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:04.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:03 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] boot 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/4142885277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]': finished 2024-09-05T01:08:05.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:08:05.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:05.203 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] boot 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/4142885277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]': finished 2024-09-05T01:08:05.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:08:05.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T01:08:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: osd.1 [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] boot 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/4142885277' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]: dispatch 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45de925c-c48d-4388-8e8d-3afcfd7c2022"}]': finished 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: osdmap e14: 3 total, 2 up, 3 in 2024-09-05T01:08:05.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:06.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:05 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/884152194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:06.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:05 smithi183 ceph-mon[34589]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:08:06.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:05 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:05 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/884152194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:05 smithi129 ceph-mon[30979]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:08:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:06.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:05 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/884152194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:06.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:05 smithi194 ceph-mon[34941]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T01:08:06.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:06 smithi183 ceph-mon[34589]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:06 smithi129 ceph-mon[30979]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:06 smithi194 ceph-mon[34941]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:08 smithi183 ceph-mon[34589]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:08 smithi129 ceph-mon[30979]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:08 smithi194 ceph-mon[34941]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:11.078 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:10 smithi194 ceph-mon[34941]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:10 smithi183 ceph-mon[34589]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:10 smithi129 ceph-mon[30979]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:12.979 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:12 smithi194 ceph-mon[34941]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:12 smithi183 ceph-mon[34589]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:12 smithi129 ceph-mon[30979]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:14.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:08:14.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:14.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:08:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T01:08:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:14.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:14 smithi194 ceph-mon[34941]: Deploying daemon osd.2 on smithi194 2024-09-05T01:08:14.927 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:14 smithi194 ceph-mon[34941]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:14 smithi183 ceph-mon[34589]: Deploying daemon osd.2 on smithi194 2024-09-05T01:08:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:14 smithi183 ceph-mon[34589]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:14 smithi129 ceph-mon[30979]: Deploying daemon osd.2 on smithi194 2024-09-05T01:08:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:14 smithi129 ceph-mon[30979]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:16 smithi183 ceph-mon[34589]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:16 smithi129 ceph-mon[30979]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:16 smithi194 ceph-mon[34941]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:18.890 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:18 smithi194 ceph-mon[34941]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:18.890 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:18.890 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:18.890 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:19.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:18 smithi129 ceph-mon[30979]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:19.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:19.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:19.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:18 smithi183 ceph-mon[34589]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:21.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:22.356 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.355+0000 7f6467fff640 1 -- 172.21.15.194:0/1318787255 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6480062630 con 0x7f64680a2b70 2024-09-05T01:08:22.357 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 2 on host 'smithi194' 2024-09-05T01:08:22.358 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 msgr2=0x7f64680a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 0x7f64680a5030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f64740314a0 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 msgr2=0x7f648019f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f647800eaa0 tx=0x7f647800ef60 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 shutdown_connections 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64680a2b70 0x7f64680a5030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f64801026d0 0x7f648019f1e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6480101d10 0x7f648019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 --2- 172.21.15.194:0/1318787255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6480100b10 0x7f648006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 >> 172.21.15.194:0/1318787255 conn(0x7f64800fc2a0 msgr2=0x7f6480105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:08:22.359 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 shutdown_connections 2024-09-05T01:08:22.360 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:08:22.358+0000 7f6485517640 1 -- 172.21.15.194:0/1318787255 wait complete. 2024-09-05T01:08:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:22 smithi194 ceph-mon[34941]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:22.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:23.009 DEBUG:teuthology.orchestra.run.smithi194:osd.2> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.2.service 2024-09-05T01:08:23.011 INFO:tasks.cephadm:Deploying osd.3 on smithi129 with /dev/vg_nvme/lv_3... 2024-09-05T01:08:23.011 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:08:23.040 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:22 smithi129 ceph-mon[30979]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:23.041 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:23.041 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:23.041 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:22 smithi183 ceph-mon[34589]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:23.269 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:08:24.369 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:08:24.172+0000 7f03972eb640 -1 osd.2 0 log_to_monitors true 2024-09-05T01:08:25.099 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:24 smithi129 ceph-mon[30979]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:25.100 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:24 smithi129 ceph-mon[30979]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.100 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:24 smithi129 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.118 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:24 smithi194 ceph-mon[34941]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:25.119 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:24 smithi194 ceph-mon[34941]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.119 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:24 smithi194 ceph-mon[34941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:24 smithi183 ceph-mon[34589]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:24 smithi183 ceph-mon[34589]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:24 smithi183 ceph-mon[34589]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T01:08:25.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:08:25.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:08:25.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:25.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:25.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:25.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:25 smithi129 ceph-mon[30979]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:08:26.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:08:26.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:08:26.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:26.062 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:26.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:26.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:25 smithi183 ceph-mon[34589]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:08:26.074 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:08:26.106 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: from='osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:08:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:25 smithi194 ceph-mon[34941]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:08:26.405 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:08:26.750 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:08:26.436+0000 7f039326e640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T01:08:26.750 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:08:26.440+0000 7f038b08e640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:08:26.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 -- 172.21.15.129:0/28478276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 msgr2=0x7f9b1410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 --2- 172.21.15.129:0/28478276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1410a050 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009ca0 tx=0x7f9b0c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 -- 172.21.15.129:0/28478276 shutdown_connections 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 --2- 172.21.15.129:0/28478276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1410a050 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 --2- 172.21.15.129:0/28478276 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14102e00 0x7f9b14103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 --2- 172.21.15.129:0/28478276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14101c00 0x7f9b14102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 -- 172.21.15.129:0/28478276 >> 172.21.15.129:0/28478276 conn(0x7f9b140fd390 msgr2=0x7f9b140ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 -- 172.21.15.129:0/28478276 shutdown_connections 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.911+0000 7f9b1a093640 1 -- 172.21.15.129:0/28478276 wait complete. 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 Processor -- start 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 -- start start 2024-09-05T01:08:26.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 0x7f9b14197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14102e00 0x7f9b14198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1410d730 con 0x7f9b141037c0 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1410d5b0 con 0x7f9b14102e00 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b1a093640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1410d8b0 con 0x7f9b14101c00 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53024/0 (socket says 172.21.15.129:53024) 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b13fff640 1 -- 172.21.15.129:0/2484496332 learned_addr learned my addr 172.21.15.129:0/2484496332 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.912+0000 7f9b12ffd640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14102e00 0x7f9b14198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b137fe640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 0x7f9b14197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 -- 172.21.15.129:0/2484496332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 msgr2=0x7f9b14197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 0x7f9b14197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 -- 172.21.15.129:0/2484496332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14102e00 msgr2=0x7f9b14198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14102e00 0x7f9b14198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 -- 172.21.15.129:0/2484496332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1419f9f0 con 0x7f9b141037c0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b137fe640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 0x7f9b14197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b13fff640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c031970 tx=0x7f9b0c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c004040 con 0x7f9b141037c0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1419fc80 con 0x7f9b141037c0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b0c004690 con 0x7f9b141037c0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c030c60 con 0x7f9b141037c0 2024-09-05T01:08:26.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.913+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b141a01c0 con 0x7f9b141037c0 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.914+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b14109050 con 0x7f9b141037c0 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.915+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f9b0c0041e0 con 0x7f9b141037c0 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.916+0000 7f9b10ff9640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 0x7f9afc0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.916+0000 7f9b137fe640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 0x7f9afc0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.916+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f9b0c0eed70 con 0x7f9b141037c0 2024-09-05T01:08:26.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.917+0000 7f9b137fe640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 0x7f9afc0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9b14102c60 tx=0x7f9b04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:26.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:26.921+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b0c0bd420 con 0x7f9b141037c0 2024-09-05T01:08:27.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:27.095+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9b14062630 con 0x7f9afc0a2b20 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi194 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.097 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='osd.2 ' entity='osd.2' 2024-09-05T01:08:27.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi194 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:27.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:27.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='osd.2 ' entity='osd.2' 2024-09-05T01:08:27.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi194 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='osd.2 ' entity='osd.2' 2024-09-05T01:08:27.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:28.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 17088M 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] boot 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:08:28.239 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 17088M 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] boot 2024-09-05T01:08:28.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:08:28.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 17088M 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: osd.2 [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] boot 2024-09-05T01:08:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T01:08:28.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T01:08:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:29.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:08:29.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:08:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T01:08:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:08:30.070 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:29 smithi129 sudo[49354]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:30.070 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:29 smithi129 sudo[49354]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:30.070 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:29 smithi129 sudo[49354]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:30.070 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:29 smithi129 sudo[49354]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:30.451 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:30 smithi129 sudo[49382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:30.451 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:30 smithi129 sudo[49382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:30.451 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:30 smithi129 sudo[49382]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:30.452 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:08:30 smithi129 sudo[49382]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:30.597 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42987]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:30.598 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42987]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:30.598 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42987]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:30.598 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42987]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]: dispatch 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]': finished 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:08:30.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:30.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:30.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:30.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2733294413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:30.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:30.638+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f9b0c0b9390 con 0x7f9b141037c0 2024-09-05T01:08:30.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:30.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T01:08:30.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]: dispatch 2024-09-05T01:08:30.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]': finished 2024-09-05T01:08:30.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:08:30.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:30.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:30.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:30.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2733294413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]: dispatch 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3184542853' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9db73173-f27a-493d-a9c8-478d042973f5"}]': finished 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: osdmap e21: 4 total, 3 up, 4 in 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:30.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:30.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:30.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2733294413' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:30.910 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:30.911 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:30.911 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:30.911 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:08:30 smithi183 sudo[42990]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:31.122 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:30 smithi194 sudo[43971]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:31.123 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:30 smithi194 sudo[43971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:31.123 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:30 smithi194 sudo[43971]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:31.123 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:30 smithi194 sudo[43971]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:31.500 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43974]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:31.500 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43974]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:31.500 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43974]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:31.501 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43974]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:31.651 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49619]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:31.651 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49619]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:31.651 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49619]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:31.651 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49619]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:31.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:31.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:31 smithi183 ceph-mon[34589]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:08:31.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:31.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:31 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49805]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49805]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 sudo[49805]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 ceph-mon[30979]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:08:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:31 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.149 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:32.149 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:32.149 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43977]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:32.149 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:31 smithi194 sudo[43977]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 sudo[43980]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 sudo[43980]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 sudo[43980]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 sudo[43980]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:32.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:32.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T01:08:32.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:32.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42993]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T01:08:32.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:32.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42993]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:32.646 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42993]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:32.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42996]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T01:08:32.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42996]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T01:08:32.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42996]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T01:08:32.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 sudo[42996]: pam_unix(sudo:session): session closed for user root 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.913 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.913 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:32 smithi183 ceph-mon[34589]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:32 smithi129 ceph-mon[30979]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: pgmap v78: 1 pgs: 1 active+clean; 449 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T01:08:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:32 smithi194 ceph-mon[34941]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T01:08:35.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:34 smithi194 ceph-mon[34941]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:34 smithi183 ceph-mon[34589]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:35.197 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:34 smithi129 ceph-mon[30979]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:36.962 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:36 smithi129 ceph-mon[30979]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-09-05T01:08:37.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:36 smithi194 ceph-mon[34941]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-09-05T01:08:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:36 smithi183 ceph-mon[34589]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 30 op/s 2024-09-05T01:08:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:38 smithi129 ceph-mon[30979]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-05T01:08:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:08:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:39.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:38 smithi194 ceph-mon[34941]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-05T01:08:39.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:08:39.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:38 smithi183 ceph-mon[34589]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-05T01:08:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T01:08:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:39.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:39 smithi129 ceph-mon[30979]: Deploying daemon osd.3 on smithi129 2024-09-05T01:08:40.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:39 smithi183 ceph-mon[34589]: Deploying daemon osd.3 on smithi129 2024-09-05T01:08:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:39 smithi194 ceph-mon[34941]: Deploying daemon osd.3 on smithi129 2024-09-05T01:08:40.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:40 smithi129 ceph-mon[30979]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-05T01:08:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:40 smithi183 ceph-mon[34589]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-05T01:08:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:40 smithi194 ceph-mon[34941]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-05T01:08:42.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:42 smithi129 ceph-mon[30979]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-05T01:08:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:42 smithi183 ceph-mon[34589]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-05T01:08:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:42 smithi194 ceph-mon[34941]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-05T01:08:44.101 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:44.101 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:44.101 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:44.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:45.084 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:44 smithi129 ceph-mon[30979]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:45.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:44 smithi183 ceph-mon[34589]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:44 smithi194 ceph-mon[34941]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:47.107 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:47.107 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.107 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.108 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:47.108 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:47.108 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-05T01:08:47.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:47.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:47.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.764+0000 7f9b10ff9640 1 -- 172.21.15.129:0/2484496332 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9b14062630 con 0x7f9afc0a2b20 2024-09-05T01:08:47.766 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 3 on host 'smithi129' 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 msgr2=0x7f9afc0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 0x7f9afc0a4fe0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9b14102c60 tx=0x7f9b04005f90 comp rx=0 tx=0).stop 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 msgr2=0x7f9b1419f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c031970 tx=0x7f9b0c003ba0 comp rx=0 tx=0).stop 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 shutdown_connections 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9afc0a2b20 0x7f9afc0a4fe0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:47.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b141037c0 0x7f9b1419f380 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b14102e00 0x7f9b14198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 --2- 172.21.15.129:0/2484496332 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9b14101c00 0x7f9b14197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 >> 172.21.15.129:0/2484496332 conn(0x7f9b140fd390 msgr2=0x7f9b14106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:08:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 shutdown_connections 2024-09-05T01:08:47.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:08:47.767+0000 7f9b1a093640 1 -- 172.21.15.129:0/2484496332 wait complete. 2024-09-05T01:08:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:47 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:47 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:47 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:47 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:47 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:47 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:47 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:08:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:47 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:47 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:48.471 DEBUG:teuthology.orchestra.run.smithi129:osd.3> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.3.service 2024-09-05T01:08:48.473 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-09-05T01:08:48.473 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:08:48.727 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:08:48.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:48 smithi129 ceph-mon[30979]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:49.116 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:48 smithi183 ceph-mon[34589]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:48 smithi194 ceph-mon[34941]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:49.781 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:49 smithi129 ceph-mon[30979]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:08:49.781 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:08:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:08:49.524+0000 7f5756494640 -1 osd.3 0 log_to_monitors true 2024-09-05T01:08:50.027 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:49 smithi183 ceph-mon[34589]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:08:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:49 smithi194 ceph-mon[34941]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T01:08:50.934 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:50 smithi129 ceph-mon[30979]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:50.934 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:50 smithi129 ceph-mon[30979]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:08:50.934 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:50 smithi129 ceph-mon[30979]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:08:50.934 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:50 smithi129 ceph-mon[30979]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:08:50.935 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:50 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:50 smithi183 ceph-mon[34589]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:50 smithi183 ceph-mon[34589]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:08:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:50 smithi183 ceph-mon[34589]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:08:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:50 smithi183 ceph-mon[34589]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:08:51.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:50 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:50 smithi194 ceph-mon[34941]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:50 smithi194 ceph-mon[34941]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T01:08:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:50 smithi194 ceph-mon[34941]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T01:08:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:50 smithi194 ceph-mon[34941]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:08:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:50 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:08:51.299 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-09-05T01:08:51.551 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:08:51.742 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:08:51.523+0000 7f5752417640 -1 osd.3 0 waiting for initial osdmap 2024-09-05T01:08:51.742 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:08:51.530+0000 7f574a237640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:51.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] boot 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:08:51.912 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:51.999+0000 7fbac1571640 1 -- 172.21.15.183:0/3418445451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 msgr2=0x7fbabc103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:51.999+0000 7fbac1571640 1 --2- 172.21.15.183:0/3418445451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc103280 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab002a410 comp rx=0 tx=0).stop 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 -- 172.21.15.183:0/3418445451 shutdown_connections 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 --2- 172.21.15.183:0/3418445451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc1037c0 0x7fbabc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 --2- 172.21.15.183:0/3418445451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc103280 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 --2- 172.21.15.183:0/3418445451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc101c00 0x7fbabc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 -- 172.21.15.183:0/3418445451 >> 172.21.15.183:0/3418445451 conn(0x7fbabc0fd390 msgr2=0x7fbabc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:08:52.031 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 -- 172.21.15.183:0/3418445451 shutdown_connections 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 -- 172.21.15.183:0/3418445451 wait complete. 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 Processor -- start 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.000+0000 7fbac1571640 1 -- start start 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 0x7fbabc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc1037c0 0x7fbabc19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc10d730 con 0x7fbabc101c00 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc10d5b0 con 0x7fbabc102e00 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbac1571640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc10d8b0 con 0x7fbabc1037c0 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:52.032 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:51470/0 (socket says 172.21.15.183:51470) 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 -- 172.21.15.183:0/1159076560 learned_addr learned my addr 172.21.15.183:0/1159076560 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbabaffd640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 0x7fbabc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbabb7fe640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc1037c0 0x7fbabc19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 -- 172.21.15.183:0/1159076560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc1037c0 msgr2=0x7fbabc19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc1037c0 0x7fbabc19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 -- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 msgr2=0x7fbabc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 0x7fbabc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 -- 172.21.15.183:0/1159076560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc19fad0 con 0x7fbabc102e00 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbabaffd640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 0x7fbabc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.001+0000 7fbaba7fc640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005940 tx=0x7fbab00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:52.033 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.002+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab003f030 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.002+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc19fd60 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.002+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab00318f0 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.002+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0035070 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.002+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc1a01e0 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.003+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc1090a0 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.004+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbab0042030 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.005+0000 7fbaa3fff640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 0x7fbaa40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.005+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3702+0+0 (secure 0 0 0) 0x7fbab00ee870 con 0x7fbabc102e00 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.005+0000 7fbabaffd640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 0x7fbaa40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.005+0000 7fbabaffd640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 0x7fbaa40a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbabc102c60 tx=0x7fbaac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:08:52.034 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.008+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbab00bc380 con 0x7fbabc102e00 2024-09-05T01:08:52.202 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:08:52.202+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbabc062630 con 0x7fbaa40a2bb0 2024-09-05T01:08:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:08:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:08:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] boot 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:08:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:08:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T01:08:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732]' entity='osd.3' 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: osd.3 [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] boot 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: osdmap e24: 4 total, 4 up, 4 in 2024-09-05T01:08:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T01:08:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:08:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:08:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi129 2024-09-05T01:08:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-05T01:08:52.850 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:52.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:52.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:52.851 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:08:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi129 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:53.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi129 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-05T01:08:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:08:53.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:08:53.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:08:53.905 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:53 smithi183 ceph-mon[34589]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:53.905 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:53 smithi183 ceph-mon[34589]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:08:54.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:53 smithi129 ceph-mon[30979]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:54.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:53 smithi129 ceph-mon[30979]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:08:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:53 smithi194 ceph-mon[34941]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:08:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:53 smithi194 ceph-mon[34941]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T01:08:55.112 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:08:55.112 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T01:08:55.112 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/4145249563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.112 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.112 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]': finished 2024-09-05T01:08:55.113 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:08:55.113 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:08:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T01:08:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/4145249563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]': finished 2024-09-05T01:08:55.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:08:55.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/4145249563' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]: dispatch 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d31dce58-e63a-47ed-8940-9db4c09d6d1f"}]': finished 2024-09-05T01:08:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: osdmap e27: 5 total, 4 up, 5 in 2024-09-05T01:08:55.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:56.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:55 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/2204929284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:56.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:55 smithi183 ceph-mon[34589]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:08:56.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:56.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:55 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/2204929284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:56.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:55 smithi129 ceph-mon[30979]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:08:56.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:56.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:55 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/2204929284' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:08:56.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:55 smithi194 ceph-mon[34941]: osdmap e28: 5 total, 4 up, 5 in 2024-09-05T01:08:56.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:08:57.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:56 smithi194 ceph-mon[34941]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:08:57.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:56 smithi183 ceph-mon[34589]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:08:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:56 smithi129 ceph-mon[30979]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:08:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:08:58 smithi183 ceph-mon[34589]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T01:08:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:08:58 smithi129 ceph-mon[30979]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T01:08:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:08:58 smithi194 ceph-mon[34941]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-05T01:09:01.002 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:00 smithi183 ceph-mon[34589]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:00 smithi129 ceph-mon[30979]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:00 smithi194 ceph-mon[34941]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:03.061 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:02 smithi183 ceph-mon[34589]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:03.061 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:09:03.061 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:02 smithi129 ceph-mon[30979]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:09:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:02 smithi194 ceph-mon[34941]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T01:09:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:04.080 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:03 smithi183 ceph-mon[34589]: Deploying daemon osd.4 on smithi183 2024-09-05T01:09:04.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:03 smithi129 ceph-mon[30979]: Deploying daemon osd.4 on smithi183 2024-09-05T01:09:04.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:03 smithi194 ceph-mon[34941]: Deploying daemon osd.4 on smithi183 2024-09-05T01:09:05.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:04 smithi183 ceph-mon[34589]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-05T01:09:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:04 smithi129 ceph-mon[30979]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-05T01:09:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:04 smithi194 ceph-mon[34941]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-05T01:09:06.857 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:06 smithi183 ceph-mon[34589]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:06 smithi129 ceph-mon[30979]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:06 smithi194 ceph-mon[34941]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:08 smithi183 ceph-mon[34589]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:08.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:08 smithi129 ceph-mon[30979]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:08 smithi194 ceph-mon[34941]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:10 smithi183 ceph-mon[34589]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:10 smithi129 ceph-mon[30979]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:10 smithi194 ceph-mon[34941]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:11.859 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-09-05T01:09:11.860 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.858+0000 7fbaa3fff640 1 -- 172.21.15.183:0/1159076560 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbabc062630 con 0x7fbaa40a2bb0 2024-09-05T01:09:11.861 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.860+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 msgr2=0x7fbaa40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.860+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 0x7fbaa40a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbabc102c60 tx=0x7fbaac005f90 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 msgr2=0x7fbabc1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbab0005940 tx=0x7fbab00042f0 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 shutdown_connections 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbaa40a2bb0 0x7fbaa40a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbabc1037c0 0x7fbabc19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbabc102e00 0x7fbabc1983e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 --2- 172.21.15.183:0/1159076560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbabc101c00 0x7fbabc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:11.862 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 >> 172.21.15.183:0/1159076560 conn(0x7fbabc0fd390 msgr2=0x7fbabc106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:09:11.863 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 shutdown_connections 2024-09-05T01:09:11.863 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:09:11.861+0000 7fbac1571640 1 -- 172.21.15.183:0/1159076560 wait complete. 2024-09-05T01:09:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.609 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.4.service 2024-09-05T01:09:12.612 INFO:tasks.cephadm:Deploying osd.5 on smithi194 with /dev/vg_nvme/lv_3... 2024-09-05T01:09:12.612 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T01:09:12.865 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:09:12.895 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:12 smithi183 ceph-mon[34589]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:12.895 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:12.895 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.895 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:12 smithi194 ceph-mon[34941]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:12.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:12.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:12.986 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:12 smithi129 ceph-mon[30979]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:13.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:13 smithi183 ceph-mon[34589]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:13.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:13 smithi183 ceph-mon[34589]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:13.911 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:09:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:09:13.644+0000 7f7f89aa4640 -1 osd.4 0 log_to_monitors true 2024-09-05T01:09:14.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:13 smithi194 ceph-mon[34941]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:14.207 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:13 smithi194 ceph-mon[34941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:13 smithi129 ceph-mon[30979]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:13 smithi129 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T01:09:15.143 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:09:14.840+0000 7f7f85a27640 -1 osd.4 0 waiting for initial osdmap 2024-09-05T01:09:15.143 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:09:14.846+0000 7f7f8184f640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:14 smithi183 ceph-mon[34589]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:15.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:09:15.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.237 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:09:15.238 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:15.238 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:14 smithi194 ceph-mon[34941]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T01:09:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: from='osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T01:09:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:15.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:14 smithi129 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:09:15.557 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:09:15.591 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_3 2024-09-05T01:09:15.842 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.098 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:16.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:16.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:16.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:16.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:16.317 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 -- 172.21.15.194:0/4263145724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c101cd0 msgr2=0x7f3f2c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:16.317 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 --2- 172.21.15.194:0/4263145724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009900 tx=0x7f3f2002a410 comp rx=0 tx=0).stop 2024-09-05T01:09:16.317 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 -- 172.21.15.194:0/4263145724 shutdown_connections 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 --2- 172.21.15.194:0/4263145724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c102690 0x7f3f2c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 --2- 172.21.15.194:0/4263145724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 --2- 172.21.15.194:0/4263145724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c100ad0 0x7f3f2c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 -- 172.21.15.194:0/4263145724 >> 172.21.15.194:0/4263145724 conn(0x7f3f2c0fc280 msgr2=0x7f3f2c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 -- 172.21.15.194:0/4263145724 shutdown_connections 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.299+0000 7f3f326a3640 1 -- 172.21.15.194:0/4263145724 wait complete. 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 Processor -- start 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 -- start start 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c100ad0 0x7f3f2c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:16.318 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c10d7b0 con 0x7f3f2c101cd0 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c10d630 con 0x7f3f2c100ad0 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f326a3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c10d930 con 0x7f3f2c102690 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f30c40640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f30c40640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49718/0 (socket says 172.21.15.194:49718) 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.300+0000 7f3f30c40640 1 -- 172.21.15.194:0/173049031 learned_addr learned my addr 172.21.15.194:0/173049031 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f2bfff640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c100ad0 0x7f3f2c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f2b7fe640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 -- 172.21.15.194:0/173049031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c100ad0 msgr2=0x7f3f2c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c100ad0 0x7f3f2c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 -- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c101cd0 msgr2=0x7f3f2c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:16.319 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 -- 172.21.15.194:0/173049031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c19fac0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f30c40640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400eaa0 tx=0x7f3f2400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2400cdb0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f24005020 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c19fdb0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f240116d0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.301+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c1a02e0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.303+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c107f10 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.303+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3f24011870 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.304+0000 7f3f297fa640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 0x7f3f140a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.304+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4101+0+0 (secure 0 0 0) 0x7f3f240ce1a0 con 0x7f3f2c102690 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.305+0000 7f3f2bfff640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 0x7f3f140a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:16.320 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.305+0000 7f3f2bfff640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 0x7f3f140a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c101b30 tx=0x7f3f1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:09:16.321 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.308+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f24016030 con 0x7f3f2c102690 2024-09-05T01:09:16.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:16.486+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3f2c062630 con 0x7f3f140a2bb0 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi183 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] boot 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:17.152 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi183 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] boot 2024-09-05T01:09:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:09:17.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:17.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:17.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:17.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi183 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 8544M 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: osd.4 [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] boot 2024-09-05T01:09:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: osdmap e31: 5 total, 5 up, 5 in 2024-09-05T01:09:17.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T01:09:17.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:17.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:17.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:17 smithi183 ceph-mon[34589]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:17 smithi183 ceph-mon[34589]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:09:18.233 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:17 smithi194 ceph-mon[34941]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:18.234 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:17 smithi194 ceph-mon[34941]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:09:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:17 smithi129 ceph-mon[30979]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:17 smithi129 ceph-mon[30979]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T01:09:19.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:18 smithi194 ceph-mon[34941]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:19.143 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:18 smithi129 ceph-mon[30979]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:19.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:18 smithi183 ceph-mon[34589]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/1201649745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]': finished 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:19.867 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:19 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/4160386103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/1201649745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]': finished 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:20.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:19 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/4160386103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/1201649745' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]: dispatch 2024-09-05T01:09:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c954db0-fb4f-4beb-99fe-abedf9d7e241"}]': finished 2024-09-05T01:09:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: osdmap e33: 6 total, 5 up, 6 in 2024-09-05T01:09:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:20.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:19 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/4160386103' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:20 smithi183 ceph-mon[34589]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:20 smithi129 ceph-mon[30979]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:20 smithi194 ceph-mon[34941]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:23.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:22 smithi183 ceph-mon[34589]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:22 smithi129 ceph-mon[30979]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:22 smithi194 ceph-mon[34941]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:24.124 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:23 smithi194 ceph-mon[34941]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:24.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:23 smithi183 ceph-mon[34589]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:23 smithi129 ceph-mon[30979]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:26.953 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:26 smithi194 ceph-mon[34941]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:27.136 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:26 smithi183 ceph-mon[34589]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:26 smithi129 ceph-mon[30979]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:28.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:09:28.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:27 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:28.080 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:09:28.081 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:27 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:28.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T01:09:28.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:27 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:29.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:28 smithi194 ceph-mon[34941]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:29.099 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:28 smithi194 ceph-mon[34941]: Deploying daemon osd.5 on smithi194 2024-09-05T01:09:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:28 smithi183 ceph-mon[34589]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:28 smithi183 ceph-mon[34589]: Deploying daemon osd.5 on smithi194 2024-09-05T01:09:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:28 smithi129 ceph-mon[30979]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:28 smithi129 ceph-mon[30979]: Deploying daemon osd.5 on smithi194 2024-09-05T01:09:30.887 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:30 smithi194 ceph-mon[34941]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:31.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:30 smithi183 ceph-mon[34589]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:30 smithi129 ceph-mon[30979]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:33.076 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:32 smithi194 ceph-mon[34941]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:32 smithi183 ceph-mon[34589]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:32 smithi129 ceph-mon[30979]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:33.895 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:33 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:33.895 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:33 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:33.896 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:33 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:34.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:33 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:34.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:33 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:34.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:33 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:33 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:33 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:33 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:34.871 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:34 smithi194 ceph-mon[34941]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:34 smithi183 ceph-mon[34589]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:34 smithi129 ceph-mon[30979]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:36.917 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:36.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:36.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:36.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:36.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:36.918 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.257 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.256+0000 7f3f297fa640 1 -- 172.21.15.194:0/173049031 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3f2c062630 con 0x7f3f140a2bb0 2024-09-05T01:09:37.258 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 5 on host 'smithi194' 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 msgr2=0x7f3f140a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 0x7f3f140a5070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c101b30 tx=0x7f3f1c008040 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 msgr2=0x7f3f2c19f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400eaa0 tx=0x7f3f2400ef60 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 shutdown_connections 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f140a2bb0 0x7f3f140a5070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f2c102690 0x7f3f2c19f450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f2c101cd0 0x7f3f2c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 --2- 172.21.15.194:0/173049031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f2c100ad0 0x7f3f2c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 >> 172.21.15.194:0/173049031 conn(0x7f3f2c0fc280 msgr2=0x7f3f2c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 shutdown_connections 2024-09-05T01:09:37.260 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:09:37.259+0000 7f3f326a3640 1 -- 172.21.15.194:0/173049031 wait complete. 2024-09-05T01:09:37.879 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:37 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:37.880 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:37 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.880 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:37 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.906 DEBUG:teuthology.orchestra.run.smithi194:osd.5> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.5.service 2024-09-05T01:09:37.908 INFO:tasks.cephadm:Deploying osd.6 on smithi129 with /dev/vg_nvme/lv_2... 2024-09-05T01:09:37.908 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:09:37.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:37 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:37.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:37 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:37.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:37 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:38.167 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:09:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:37 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:37 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:37 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:38.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:38 smithi129 ceph-mon[30979]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:39.128 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:38 smithi194 ceph-mon[34941]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:39.128 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:09:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:09:38.919+0000 7fb3aa9f7640 -1 osd.5 0 log_to_monitors true 2024-09-05T01:09:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:38 smithi183 ceph-mon[34589]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:39.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:39 smithi129 ceph-mon[30979]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:39.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:39 smithi129 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:40.126 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:39 smithi194 ceph-mon[34941]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:40.127 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:39 smithi194 ceph-mon[34941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:40.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:39 smithi183 ceph-mon[34589]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:40.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:39 smithi183 ceph-mon[34589]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T01:09:40.813 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:09:40.840 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-09-05T01:09:41.128 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:09:41.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:09:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:09:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:40 smithi183 ceph-mon[34589]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:40 smithi129 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: from='osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:40 smithi194 ceph-mon[34941]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.662+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/2254019265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 msgr2=0x7fb3ac101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.662+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/2254019265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac101200 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009900 tx=0x7fb39c02a410 comp rx=0 tx=0).stop 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/2254019265 shutdown_connections 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/2254019265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac108fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/2254019265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac101200 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/2254019265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac102520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/2254019265 >> 172.21.15.129:0/2254019265 conn(0x7fb3ac0fba40 msgr2=0x7fb3ac0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:09:41.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/2254019265 shutdown_connections 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/2254019265 wait complete. 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 Processor -- start 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.663+0000 7fb3b3f4a640 1 -- start start 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac10d870 con 0x7fb3ac100d80 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac10d6f0 con 0x7fb3ac102120 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac10d9f0 con 0x7fb3ac101740 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58642/0 (socket says 172.21.15.129:58642) 2024-09-05T01:09:41.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 -- 172.21.15.129:0/3500521077 learned_addr learned my addr 172.21.15.129:0/3500521077 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b14e5640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b24e7640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 -- 172.21.15.129:0/3500521077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 msgr2=0x7fb3ac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 -- 172.21.15.129:0/3500521077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 msgr2=0x7fb3ac19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 -- 172.21.15.129:0/3500521077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac19fca0 con 0x7fb3ac100d80 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b24e7640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b14e5640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b1ce6640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b450 tx=0x7fb3a800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:09:41.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8015020 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3a8011d40 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac19ff90 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800f340 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.664+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac1a0460 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.665+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3ac1080c0 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.666+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb3a8018020 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.667+0000 7fb3a2ffd640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 0x7fb3980a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.667+0000 7fb3b14e5640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 0x7fb3980a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.668+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4253+0+0 (secure 0 0 0) 0x7fb3a80ce1c0 con 0x7fb3ac100d80 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.668+0000 7fb3b14e5640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 0x7fb3980a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000c00 tx=0x7fb39c0314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:09:41.683 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.671+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3a801f020 con 0x7fb3ac100d80 2024-09-05T01:09:41.750 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:09:41.443+0000 7fb3a718e640 -1 osd.5 0 waiting for initial osdmap 2024-09-05T01:09:41.750 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:09:41.448+0000 7fb39e79a640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:09:41.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:09:41.853+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb3ac062630 con 0x7fb3980a2bb0 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:41 smithi183 ceph-mon[34589]: from='osd.5 ' entity='osd.5' 2024-09-05T01:09:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:41 smithi129 ceph-mon[30979]: from='osd.5 ' entity='osd.5' 2024-09-05T01:09:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:09:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T01:09:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:42.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:41 smithi194 ceph-mon[34941]: from='osd.5 ' entity='osd.5' 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi194 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 8544M 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] boot 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:43.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:42 smithi183 ceph-mon[34589]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:09:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:09:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:09:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi194 2024-09-05T01:09:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 8544M 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] boot 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:43.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:42 smithi129 ceph-mon[30979]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi194 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 8544M 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-05T01:09:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: osd.5 [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] boot 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: osdmap e36: 6 total, 6 up, 6 in 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:43.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:42 smithi194 ceph-mon[34941]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]: dispatch 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]': finished 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:09:44.910 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:44 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]: dispatch 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]': finished 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:09:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:44 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T01:09:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]: dispatch 2024-09-05T01:09:45.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/220507263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0213e2f7-1e56-4c20-9122-cd81c2dc4996"}]': finished 2024-09-05T01:09:45.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: osdmap e39: 7 total, 6 up, 7 in 2024-09-05T01:09:45.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:44 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:46.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:45 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3138618249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:46.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:45 smithi183 ceph-mon[34589]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:09:46.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:45 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:45 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3138618249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:45 smithi129 ceph-mon[30979]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:09:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:45 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:45 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3138618249' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:09:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:45 smithi194 ceph-mon[34941]: osdmap e40: 7 total, 6 up, 7 in 2024-09-05T01:09:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:45 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:09:47.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:46 smithi183 ceph-mon[34589]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:09:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:46 smithi129 ceph-mon[30979]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:09:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:46 smithi194 ceph-mon[34941]: pgmap v134: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-09-05T01:09:49.149 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:48 smithi129 ceph-mon[30979]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-05T01:09:49.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:48 smithi183 ceph-mon[34589]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-05T01:09:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:48 smithi194 ceph-mon[34941]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-05T01:09:50.972 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:50 smithi129 ceph-mon[30979]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:50 smithi183 ceph-mon[34589]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:50 smithi194 ceph-mon[34941]: pgmap v136: 1 pgs: 1 peering; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T01:09:53.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:52 smithi183 ceph-mon[34589]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:52 smithi129 ceph-mon[30979]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:52 smithi194 ceph-mon[34941]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-05T01:09:54.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:53 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:09:54.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:53 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:54.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:53 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:09:54.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:53 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:53 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T01:09:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:53 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:09:55.067 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:54 smithi129 ceph-mon[30979]: Deploying daemon osd.6 on smithi129 2024-09-05T01:09:55.067 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:54 smithi129 ceph-mon[30979]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T01:09:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:54 smithi183 ceph-mon[34589]: Deploying daemon osd.6 on smithi129 2024-09-05T01:09:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:54 smithi183 ceph-mon[34589]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T01:09:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:54 smithi194 ceph-mon[34941]: Deploying daemon osd.6 on smithi129 2024-09-05T01:09:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:54 smithi194 ceph-mon[34941]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-05T01:09:56.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:56 smithi129 ceph-mon[30979]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:57.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:56 smithi183 ceph-mon[34589]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:56 smithi194 ceph-mon[34941]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:58 smithi183 ceph-mon[34589]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:09:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:59.163 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:58 smithi129 ceph-mon[30979]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:59.163 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:59.163 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:59.163 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:09:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:58 smithi194 ceph-mon[34941]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:09:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:09:59.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:09:59.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:09:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:00.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:00 smithi129 ceph-mon[30979]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:00.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:00 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:10:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:00 smithi183 ceph-mon[34589]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:00 smithi183 ceph-mon[34589]: overall HEALTH_OK 2024-09-05T01:10:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:00 smithi194 ceph-mon[34941]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:00 smithi194 ceph-mon[34941]: overall HEALTH_OK 2024-09-05T01:10:02.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.417+0000 7fb3a2ffd640 1 -- 172.21.15.129:0/3500521077 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb3ac062630 con 0x7fb3980a2bb0 2024-09-05T01:10:02.418 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 6 on host 'smithi129' 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 msgr2=0x7fb3980a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 0x7fb3980a5070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb39c000c00 tx=0x7fb39c0314a0 comp rx=0 tx=0).stop 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 msgr2=0x7fb3ac198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800b450 tx=0x7fb3a800b910 comp rx=0 tx=0).stop 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 shutdown_connections 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb3980a2bb0 0x7fb3980a5070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:02.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3ac102120 0x7fb3ac19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb3ac101740 0x7fb3ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 --2- 172.21.15.129:0/3500521077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3ac100d80 0x7fb3ac198070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 >> 172.21.15.129:0/3500521077 conn(0x7fb3ac0fba40 msgr2=0x7fb3ac105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 shutdown_connections 2024-09-05T01:10:02.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:02.419+0000 7fb3b3f4a640 1 -- 172.21.15.129:0/3500521077 wait complete. 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:02.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:02.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:02.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.110 DEBUG:teuthology.orchestra.run.smithi129:osd.6> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.6.service 2024-09-05T01:10:03.113 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-09-05T01:10:03.113 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:10:03.373 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:10:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:03 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:03 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:03 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:03 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:03.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:03 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:03 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:03 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:03.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:03 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:03.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:03 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:04.580 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:04 smithi129 ceph-mon[30979]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:04.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:04 smithi183 ceph-mon[34589]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:04.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:04 smithi194 ceph-mon[34941]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:04.993 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:10:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:10:04.761+0000 7f23f9d13640 -1 osd.6 0 log_to_monitors true 2024-09-05T01:10:05.599 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:05 smithi183 ceph-mon[34589]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:10:05.667 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:05 smithi129 ceph-mon[30979]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:10:05.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:05 smithi194 ceph-mon[34941]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T01:10:06.001 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:10:06.034 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-09-05T01:10:06.287 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:06.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:06.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:06.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T01:10:06.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: osdmap e41: 7 total, 6 up, 7 in 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:06.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.775+0000 7fc91344e640 1 -- 172.21.15.183:0/3861515075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c101c00 msgr2=0x7fc90c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.775+0000 7fc91344e640 1 --2- 172.21.15.183:0/3861515075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c101c00 0x7fc90c102000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc908009900 tx=0x7fc90802a410 comp rx=0 tx=0).stop 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- 172.21.15.183:0/3861515075 shutdown_connections 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- 172.21.15.183:0/3861515075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c1037c0 0x7fc90c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- 172.21.15.183:0/3861515075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c102e00 0x7fc90c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:06.810 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- 172.21.15.183:0/3861515075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c101c00 0x7fc90c102000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- 172.21.15.183:0/3861515075 >> 172.21.15.183:0/3861515075 conn(0x7fc90c0fd390 msgr2=0x7fc90c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- 172.21.15.183:0/3861515075 shutdown_connections 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- 172.21.15.183:0/3861515075 wait complete. 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 Processor -- start 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- start start 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c102e00 0x7fc90c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 0x7fc90c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c10d710 con 0x7fc90c101c00 2024-09-05T01:10:06.811 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c10d590 con 0x7fc90c1037c0 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc91344e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c10d890 con 0x7fc90c102e00 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.776+0000 7fc9111ea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9119eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 0x7fc90c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.183:50306/0 (socket says 172.21.15.183:50306) 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 -- 172.21.15.183:0/2827522448 learned_addr learned my addr 172.21.15.183:0/2827522448 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9109e9640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c102e00 0x7fc90c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 -- 172.21.15.183:0/2827522448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c102e00 msgr2=0x7fc90c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c102e00 0x7fc90c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 -- 172.21.15.183:0/2827522448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 msgr2=0x7fc90c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 0x7fc90c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:06.812 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 -- 172.21.15.183:0/2827522448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0753e0 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9119eb640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 0x7fc90c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc9111ea640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc9080314e0 tx=0x7fc9080042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90803f030 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc908003cb0 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c1a1f00 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc908035070 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.777+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1a22a0 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.778+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c109050 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.780+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc908042030 con 0x7fc90c101c00 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.780+0000 7fc8fe7fc640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.780+0000 7fc9109e9640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:06.813 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.780+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7fc9080eec40 con 0x7fc90c101c00 2024-09-05T01:10:06.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.781+0000 7fc9109e9640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 0x7fc8f80a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc90c075d10 tx=0x7fc90000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:06.814 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.788+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9080bc2e0 con 0x7fc90c101c00 2024-09-05T01:10:06.964 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:06.963+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc90c062630 con 0x7fc8f80a2b80 2024-09-05T01:10:07.404 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:10:07.038+0000 7f23f5c96640 -1 osd.6 0 waiting for initial osdmap 2024-09-05T01:10:07.404 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:10:07.049+0000 7f23edab6640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi129 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.655 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:07.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:07.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:07.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' 2024-09-05T01:10:07.656 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:07 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi129 2024-09-05T01:10:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-05T01:10:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:10:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:10:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' 2024-09-05T01:10:07.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:07 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi129 2024-09-05T01:10:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-05T01:10:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:10:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T01:10:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741]' entity='osd.6' 2024-09-05T01:10:07.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:07 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:08.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:10:08.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:10:08.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:08.676 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:08.677 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] boot 2024-09-05T01:10:08.677 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:10:08.677 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] boot 2024-09-05T01:10:08.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:10:08.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: from='client.14391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: osd.6 [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] boot 2024-09-05T01:10:08.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: osdmap e43: 7 total, 7 up, 7 in 2024-09-05T01:10:08.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T01:10:10.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:10:10.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/2797193648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]': finished 2024-09-05T01:10:10.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:10:10.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:10.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:10.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:10 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/551955866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:10.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:10:10.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/2797193648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]': finished 2024-09-05T01:10:10.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:10:10.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:10.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:10.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:10 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/551955866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: osdmap e44: 7 total, 7 up, 7 in 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/2797193648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]: dispatch 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd9aefa5-14e5-492a-8eb0-a0704322665a"}]': finished 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: osdmap e45: 8 total, 7 up, 8 in 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:10.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:10 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/551955866' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:12 smithi183 ceph-mon[34589]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:12 smithi129 ceph-mon[30979]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:12 smithi194 ceph-mon[34941]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:15.144 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:14 smithi183 ceph-mon[34589]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:14 smithi129 ceph-mon[30979]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:14 smithi194 ceph-mon[34941]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:16.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:16 smithi183 ceph-mon[34589]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:16 smithi129 ceph-mon[30979]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:17.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:16 smithi194 ceph-mon[34941]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:19.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:18 smithi129 ceph-mon[30979]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:19.045 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:10:19.045 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:18 smithi183 ceph-mon[34589]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:10:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:18 smithi194 ceph-mon[34941]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T01:10:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:20.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:19 smithi183 ceph-mon[34589]: Deploying daemon osd.7 on smithi183 2024-09-05T01:10:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:19 smithi129 ceph-mon[30979]: Deploying daemon osd.7 on smithi183 2024-09-05T01:10:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:19 smithi194 ceph-mon[34941]: Deploying daemon osd.7 on smithi183 2024-09-05T01:10:21.055 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:20 smithi183 ceph-mon[34589]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:20 smithi129 ceph-mon[30979]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:20 smithi194 ceph-mon[34941]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:22.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:22 smithi183 ceph-mon[34589]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:22 smithi129 ceph-mon[30979]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:22 smithi194 ceph-mon[34941]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:24.045 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:24.045 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:24.045 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:25.064 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:24 smithi183 ceph-mon[34589]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:24 smithi129 ceph-mon[30979]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:24 smithi194 ceph-mon[34941]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:27.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:26 smithi194 ceph-mon[34941]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:27.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.053 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.073 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:26 smithi183 ceph-mon[34589]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:27.073 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.073 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.114 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-09-05T01:10:27.115 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.113+0000 7fc8fe7fc640 1 -- 172.21.15.183:0/2827522448 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc90c062630 con 0x7fc8f80a2b80 2024-09-05T01:10:27.117 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.116+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 msgr2=0x7fc8f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.116+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 0x7fc8f80a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc90c075d10 tx=0x7fc90000b390 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 msgr2=0x7fc90c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc9080314e0 tx=0x7fc9080042e0 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 shutdown_connections 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc8f80a2b80 0x7fc8f80a5040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90c1037c0 0x7fc90c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc90c102e00 0x7fc90c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 --2- 172.21.15.183:0/2827522448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc90c101c00 0x7fc90c075f90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 >> 172.21.15.183:0/2827522448 conn(0x7fc90c0fd390 msgr2=0x7fc90c106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 shutdown_connections 2024-09-05T01:10:27.118 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:10:27.117+0000 7fc91344e640 1 -- 172.21.15.183:0/2827522448 wait complete. 2024-09-05T01:10:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:26 smithi129 ceph-mon[30979]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:27.774 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.7.service 2024-09-05T01:10:27.776 INFO:tasks.cephadm:Deploying osd.8 on smithi194 with /dev/vg_nvme/lv_2... 2024-09-05T01:10:27.776 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T01:10:28.031 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:10:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.402 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:28 smithi194 ceph-mon[34941]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:28.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:28 smithi183 ceph-mon[34589]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:28 smithi129 ceph-mon[30979]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:29.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:29 smithi183 ceph-mon[34589]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:29.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:29 smithi183 ceph-mon[34589]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:29.911 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:10:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:10:29.650+0000 7fc6291da640 -1 osd.7 0 log_to_monitors true 2024-09-05T01:10:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:29 smithi194 ceph-mon[34941]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:29.981 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:29 smithi194 ceph-mon[34941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:29.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:29 smithi129 ceph-mon[30979]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:29.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:29 smithi129 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T01:10:30.767 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:10:30.794 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_2 2024-09-05T01:10:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:10:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:10:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:30.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:30 smithi129 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:31.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:30 smithi194 ceph-mon[34941]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:31.045 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: osdmap e46: 8 total, 7 up, 8 in 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: from='osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:30 smithi183 ceph-mon[34589]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 -- 172.21.15.194:0/1986390204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c102660 msgr2=0x7fa60c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 --2- 172.21.15.194:0/1986390204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c102660 0x7fa60c104a50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa600009900 tx=0x7fa60002a410 comp rx=0 tx=0).stop 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 -- 172.21.15.194:0/1986390204 shutdown_connections 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 --2- 172.21.15.194:0/1986390204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c1078c0 0x7fa60c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 --2- 172.21.15.194:0/1986390204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c104f90 0x7fa60c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 --2- 172.21.15.194:0/1986390204 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c102660 0x7fa60c104a50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 -- 172.21.15.194:0/1986390204 >> 172.21.15.194:0/1986390204 conn(0x7fa60c0fc280 msgr2=0x7fa60c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:31.560 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 -- 172.21.15.194:0/1986390204 shutdown_connections 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.542+0000 7fa6109da640 1 -- 172.21.15.194:0/1986390204 wait complete. 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 Processor -- start 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 -- start start 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 0x7fa60c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10d790 con 0x7fa60c104f90 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10d610 con 0x7fa60c102660 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa6109da640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c10d910 con 0x7fa60c1078c0 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.543+0000 7fa60ad9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:31.561 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60ad9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:39672/0 (socket says 172.21.15.194:39672) 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.194:56608/0 (socket says 172.21.15.194:56608) 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60ad9d640 1 -- 172.21.15.194:0/1689681452 learned_addr learned my addr 172.21.15.194:0/1689681452 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa609d9b640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 0x7fa60c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 -- 172.21.15.194:0/1689681452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 msgr2=0x7fa60c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 -- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 msgr2=0x7fa60c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 0x7fa60c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:31.562 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 -- 172.21.15.194:0/1689681452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c19faa0 con 0x7fa60c102660 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60ad9d640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa609d9b640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 0x7fa60c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa60a59c640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa60002a930 tx=0x7fa600003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60003f070 con 0x7fa60c102660 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa600031c40 con 0x7fa60c102660 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c19fd30 con 0x7fa60c102660 2024-09-05T01:10:31.563 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600035030 con 0x7fa60c102660 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.544+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c1a0190 con 0x7fa60c102660 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.545+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60c0fdd70 con 0x7fa60c102660 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.546+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa600042070 con 0x7fa60c102660 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.547+0000 7fa5fb7fe640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 0x7fa5f40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.547+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5117+0+0 (secure 0 0 0) 0x7fa6000eef30 con 0x7fa60c102660 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.547+0000 7fa609d9b640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 0x7fa5f40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.548+0000 7fa609d9b640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 0x7fa5f40a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa60c199390 tx=0x7fa5fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:31.564 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.550+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa60011e1b0 con 0x7fa60c102660 2024-09-05T01:10:31.660 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:10:31.394+0000 7fc62515d640 -1 osd.7 0 waiting for initial osdmap 2024-09-05T01:10:31.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:10:31.402+0000 7fc61cf7d640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:10:31.730 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:31.729+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa60c062630 con 0x7fa5f40a2bb0 2024-09-05T01:10:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:10:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi183 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:31 smithi129 ceph-mon[30979]: from='osd.7 ' entity='osd.7' 2024-09-05T01:10:32.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:10:32.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi183 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:31 smithi194 ceph-mon[34941]: from='osd.7 ' entity='osd.7' 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi183 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 5696M 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:31 smithi183 ceph-mon[34589]: from='osd.7 ' entity='osd.7' 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] boot 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:10:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:32 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] boot 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:32 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: osd.7 [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] boot 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: osdmap e48: 8 total, 8 up, 8 in 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:32 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:33.907 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:33 smithi194 ceph-mon[34941]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:10:34.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:33 smithi183 ceph-mon[34589]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:10:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:33 smithi129 ceph-mon[30979]: osdmap e49: 8 total, 8 up, 8 in 2024-09-05T01:10:35.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/1952531076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]': finished 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T01:10:35.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/1952531076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]': finished 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T01:10:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/1952531076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]: dispatch 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f64d8289-4220-4110-b3f0-b17294db4107"}]': finished 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: osdmap e51: 9 total, 8 up, 9 in 2024-09-05T01:10:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:36.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:35 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/255933172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:36.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:35 smithi194 ceph-mon[34941]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:10:36.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:35 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:36.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:35 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/255933172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:36.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:35 smithi183 ceph-mon[34589]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:10:36.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:35 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:35 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/255933172' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:10:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:35 smithi129 ceph-mon[30979]: osdmap e52: 9 total, 8 up, 9 in 2024-09-05T01:10:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:35 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:37.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:36 smithi194 ceph-mon[34941]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:37.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:36 smithi183 ceph-mon[34589]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:36 smithi129 ceph-mon[30979]: pgmap v171: 1 pgs: 1 remapped+peering; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:39.119 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:38 smithi194 ceph-mon[34941]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:38 smithi183 ceph-mon[34589]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:38 smithi129 ceph-mon[30979]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:41.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:40 smithi194 ceph-mon[34941]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:41.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:40 smithi183 ceph-mon[34589]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:40 smithi129 ceph-mon[30979]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:43.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:42 smithi194 ceph-mon[34941]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:43.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:10:43.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:42 smithi183 ceph-mon[34589]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:10:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:42 smithi129 ceph-mon[30979]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T01:10:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:44.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:43 smithi183 ceph-mon[34589]: Deploying daemon osd.8 on smithi194 2024-09-05T01:10:44.212 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:43 smithi194 ceph-mon[34941]: Deploying daemon osd.8 on smithi194 2024-09-05T01:10:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:43 smithi129 ceph-mon[30979]: Deploying daemon osd.8 on smithi194 2024-09-05T01:10:44.826 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:44 smithi194 ceph-mon[34941]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:44 smithi183 ceph-mon[34589]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:44 smithi129 ceph-mon[30979]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:47.150 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:46 smithi194 ceph-mon[34941]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:47.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:46 smithi183 ceph-mon[34589]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:46 smithi129 ceph-mon[30979]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:49.030 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:48 smithi194 ceph-mon[34941]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:49.030 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:49.030 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:49.030 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:49.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:48 smithi129 ceph-mon[30979]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:49.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:49.044 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:49.045 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:49.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:48 smithi183 ceph-mon[34589]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:50 smithi183 ceph-mon[34589]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:50 smithi129 ceph-mon[30979]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:50 smithi194 ceph-mon[34941]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:52.101 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 8 on host 'smithi194' 2024-09-05T01:10:52.102 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.099+0000 7fa5fb7fe640 1 -- 172.21.15.194:0/1689681452 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa60c062630 con 0x7fa5f40a2bb0 2024-09-05T01:10:52.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 msgr2=0x7fa5f40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:52.103 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 0x7fa5f40a5070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa60c199390 tx=0x7fa5fc005f90 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 msgr2=0x7fa60c197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa60002a930 tx=0x7fa600003c10 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 shutdown_connections 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5f40a2bb0 0x7fa5f40a5070 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa60c1078c0 0x7fa60c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa60c104f90 0x7fa60c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 --2- 172.21.15.194:0/1689681452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa60c102660 0x7fa60c197e70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 >> 172.21.15.194:0/1689681452 conn(0x7fa60c0fc280 msgr2=0x7fa60c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 shutdown_connections 2024-09-05T01:10:52.104 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:10:52.102+0000 7fa6109da640 1 -- 172.21.15.194:0/1689681452 wait complete. 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:52 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.706 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.707 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:52 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.732 DEBUG:teuthology.orchestra.run.smithi194:osd.8> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.8.service 2024-09-05T01:10:52.735 INFO:tasks.cephadm:Deploying osd.9 on smithi129 with /dev/vg_nvme/lv_1... 2024-09-05T01:10:52.735 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:10:52.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:52.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:52.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:52.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:10:52.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.744 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:52 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:52.994 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:10:54.238 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:10:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:10:54.197+0000 7fe643b42640 -1 osd.8 0 log_to_monitors true 2024-09-05T01:10:54.981 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:54 smithi129 ceph-mon[30979]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:54.981 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:54 smithi129 ceph-mon[30979]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:54.981 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:54 smithi129 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:55.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:54 smithi183 ceph-mon[34589]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:54 smithi183 ceph-mon[34589]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:54 smithi183 ceph-mon[34589]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:54 smithi194 ceph-mon[34941]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:54 smithi194 ceph-mon[34941]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:54 smithi194 ceph-mon[34941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T01:10:55.775 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:10:55.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-09-05T01:10:56.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:10:56.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:10:56.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:56.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:55 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.063 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.064 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.065 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:56.065 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:56.065 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:55 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.096 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:10:56.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T01:10:56.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T01:10:56.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:56.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:10:56.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:55 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:10:56.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.602+0000 7f5657025640 1 -- 172.21.15.129:0/2327781377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650101cd0 msgr2=0x7f5650102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:56.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.602+0000 7f5657025640 1 --2- 172.21.15.129:0/2327781377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650101cd0 0x7f5650102150 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f563c009900 tx=0x7f563c02a410 comp rx=0 tx=0).stop 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 -- 172.21.15.129:0/2327781377 shutdown_connections 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 --2- 172.21.15.129:0/2327781377 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650102690 0x7f5650108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 --2- 172.21.15.129:0/2327781377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650101cd0 0x7f5650102150 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 --2- 172.21.15.129:0/2327781377 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 0x7f5650100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 -- 172.21.15.129:0/2327781377 >> 172.21.15.129:0/2327781377 conn(0x7f56500fc280 msgr2=0x7f56500fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 -- 172.21.15.129:0/2327781377 shutdown_connections 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 -- 172.21.15.129:0/2327781377 wait complete. 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 Processor -- start 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.603+0000 7f5657025640 1 -- start start 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 0x7f5650197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650101cd0 0x7f5650198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:56.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f565010d730 con 0x7f5650102690 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f565010d5b0 con 0x7f5650101cd0 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f565010d8b0 con 0x7f5650100ad0 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34616/0 (socket says 172.21.15.129:34616) 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f564ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650101cd0 0x7f5650198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 -- 172.21.15.129:0/1791111261 learned_addr learned my addr 172.21.15.129:0/1791111261 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5654dc1640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 0x7f5650197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 -- 172.21.15.129:0/1791111261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 msgr2=0x7f5650197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:56.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 0x7f5650197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 -- 172.21.15.129:0/1791111261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650101cd0 msgr2=0x7f5650198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650101cd0 0x7f5650198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 -- 172.21.15.129:0/1791111261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565019fa10 con 0x7f5650102690 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f56555c2640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f564800cad0 tx=0x7f564800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800ae20 con 0x7f5650102690 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5648005020 con 0x7f5650102690 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565019fd00 con 0x7f5650102690 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.604+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800f6b0 con 0x7f5650102690 2024-09-05T01:10:56.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.605+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56501a01c0 con 0x7f5650102690 2024-09-05T01:10:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.606+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5650107f20 con 0x7f5650102690 2024-09-05T01:10:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.607+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f56480051d0 con 0x7f5650102690 2024-09-05T01:10:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.607+0000 7f564dffb640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 0x7f56400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:10:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.607+0000 7f5654dc1640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 0x7f56400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:10:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.607+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5549+0+0 (secure 0 0 0) 0x7f56480ce4c0 con 0x7f5650102690 2024-09-05T01:10:56.611 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.608+0000 7f5654dc1640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 0x7f56400a50f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5650101b30 tx=0x7f5644005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:10:56.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.611+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5648014030 con 0x7f5650102690 2024-09-05T01:10:56.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:10:56.476+0000 7fe6402d9640 -1 osd.8 0 waiting for initial osdmap 2024-09-05T01:10:56.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:10:56.486+0000 7fe6378e5640 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:10:56.789 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:10:56.787+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5650062630 con 0x7f56400a2c30 2024-09-05T01:10:57.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:57.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi194 2024-09-05T01:10:57.082 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 5696M 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: from='osd.8 ' entity='osd.8' 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] boot 2024-09-05T01:10:57.083 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:10:57.084 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:57.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi194 2024-09-05T01:10:57.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 5696M 2024-09-05T01:10:57.195 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:10:57.195 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:10:57.195 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.195 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.195 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: from='osd.8 ' entity='osd.8' 2024-09-05T01:10:57.196 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] boot 2024-09-05T01:10:57.196 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:10:57.196 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-09-05T01:10:57.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi194 2024-09-05T01:10:57.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 5696M 2024-09-05T01:10:57.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:10:57.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T01:10:57.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: from='osd.8 ' entity='osd.8' 2024-09-05T01:10:57.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: osd.8 [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] boot 2024-09-05T01:10:57.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: osdmap e55: 9 total, 9 up, 9 in 2024-09-05T01:10:57.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:10:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:10:58.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:10:58.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:10:58.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:10:58.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:10:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:58 smithi183 ceph-mon[34589]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:10:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:58 smithi183 ceph-mon[34589]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:10:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:58 smithi183 ceph-mon[34589]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:10:59.171 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:58 smithi129 ceph-mon[30979]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:10:59.171 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:58 smithi129 ceph-mon[30979]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:10:59.171 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:58 smithi129 ceph-mon[30979]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:10:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:58 smithi194 ceph-mon[34941]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:10:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:58 smithi194 ceph-mon[34941]: osdmap e56: 9 total, 9 up, 9 in 2024-09-05T01:10:59.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:58 smithi194 ceph-mon[34941]: osdmap e57: 9 total, 9 up, 9 in 2024-09-05T01:10:59.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:59 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]: dispatch 2024-09-05T01:10:59.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:59 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]': finished 2024-09-05T01:10:59.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:59 smithi129 ceph-mon[30979]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T01:10:59.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:10:59 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:00.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:59 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]: dispatch 2024-09-05T01:11:00.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:59 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]': finished 2024-09-05T01:11:00.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:59 smithi183 ceph-mon[34589]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T01:11:00.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:10:59 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:00.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:59 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]: dispatch 2024-09-05T01:11:00.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:59 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1179816380' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d297ed4-17e8-44e6-a4a2-70390bf762ab"}]': finished 2024-09-05T01:11:00.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:59 smithi194 ceph-mon[34941]: osdmap e58: 10 total, 9 up, 10 in 2024-09-05T01:11:00.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:10:59 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:01.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:00 smithi183 ceph-mon[34589]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:00 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2015176005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:00 smithi183 ceph-mon[34589]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:11:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:00 smithi129 ceph-mon[30979]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:00 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2015176005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:00 smithi129 ceph-mon[30979]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:11:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:00 smithi194 ceph-mon[34941]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:00 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2015176005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:00 smithi194 ceph-mon[34941]: osdmap e59: 10 total, 9 up, 10 in 2024-09-05T01:11:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:02 smithi183 ceph-mon[34589]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:02 smithi129 ceph-mon[30979]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:02 smithi194 ceph-mon[34941]: pgmap v191: 1 pgs: 1 peering; 577 KiB data, 660 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:04 smithi129 ceph-mon[30979]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:11:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:04 smithi183 ceph-mon[34589]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:11:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:04 smithi194 ceph-mon[34941]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T01:11:06.984 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:06 smithi129 ceph-mon[30979]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:11:07.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:06 smithi183 ceph-mon[34589]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:11:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:06 smithi194 ceph-mon[34941]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T01:11:08.876 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:08 smithi129 ceph-mon[30979]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:11:08.876 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:11:08.877 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:08 smithi183 ceph-mon[34589]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:11:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:11:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:08 smithi194 ceph-mon[34941]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-05T01:11:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T01:11:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:10.098 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:09 smithi129 ceph-mon[30979]: Deploying daemon osd.9 on smithi129 2024-09-05T01:11:10.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:09 smithi183 ceph-mon[34589]: Deploying daemon osd.9 on smithi129 2024-09-05T01:11:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:09 smithi194 ceph-mon[34941]: Deploying daemon osd.9 on smithi129 2024-09-05T01:11:11.101 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:10 smithi129 ceph-mon[30979]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:11:11.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:10 smithi183 ceph-mon[34589]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:11:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:10 smithi194 ceph-mon[34941]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T01:11:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:12 smithi183 ceph-mon[34589]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T01:11:13.182 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:12 smithi129 ceph-mon[30979]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T01:11:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:12 smithi194 ceph-mon[34941]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-05T01:11:14.932 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:14 smithi129 ceph-mon[30979]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:11:14.933 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:14.933 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:14.933 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:14 smithi183 ceph-mon[34589]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:11:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:14 smithi194 ceph-mon[34941]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:11:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:16.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:16 smithi129 ceph-mon[30979]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:16 smithi183 ceph-mon[34589]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:16 smithi194 ceph-mon[34941]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:17.720 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 9 on host 'smithi129' 2024-09-05T01:11:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.718+0000 7f564dffb640 1 -- 172.21.15.129:0/1791111261 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5650062630 con 0x7f56400a2c30 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 msgr2=0x7f56400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 0x7f56400a50f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5650101b30 tx=0x7f5644005f90 comp rx=0 tx=0).stop 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 msgr2=0x7f565019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f564800cad0 tx=0x7f564800cf90 comp rx=0 tx=0).stop 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 shutdown_connections 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56400a2c30 0x7f56400a50f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5650102690 0x7f565019f3a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5650101cd0 0x7f5650198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 --2- 172.21.15.129:0/1791111261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5650100ad0 0x7f5650197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 >> 172.21.15.129:0/1791111261 conn(0x7f56500fc280 msgr2=0x7f56501050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:11:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 shutdown_connections 2024-09-05T01:11:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:11:17.721+0000 7f5657025640 1 -- 172.21.15.129:0/1791111261 wait complete. 2024-09-05T01:11:18.534 DEBUG:teuthology.orchestra.run.smithi129:osd.9> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.9.service 2024-09-05T01:11:18.537 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-09-05T01:11:18.537 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:11:18.796 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:11:18.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:18.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:18.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:18 smithi183 ceph-mon[34589]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:18.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:18.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:18.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:18 smithi129 ceph-mon[30979]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:19.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:19.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:19.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:18 smithi194 ceph-mon[34941]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:19.842 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:19.842 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:20.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:20.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:20.022 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:20.022 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:20.550 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:11:20 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:11:20.266+0000 7f2813da7640 -1 osd.9 0 log_to_monitors true 2024-09-05T01:11:20.926 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:20 smithi129 ceph-mon[30979]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:20.926 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:20 smithi129 ceph-mon[30979]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:11:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:20 smithi183 ceph-mon[34589]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:20 smithi183 ceph-mon[34589]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:11:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:20 smithi194 ceph-mon[34941]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:20 smithi194 ceph-mon[34941]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T01:11:21.575 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-09-05T01:11:21.602 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-09-05T01:11:21.829 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-mon[30979]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:11:21.829 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-mon[30979]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:11:21.829 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-mon[30979]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:11:21.829 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:21.829 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-mon[30979]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:11:21.830 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:11:21.756+0000 7f280fd2a640 -1 osd.9 0 waiting for initial osdmap 2024-09-05T01:11:21.830 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:11:21 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:11:21.775+0000 7f280bb52640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:11:21.854 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:11:21.887 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:21 smithi183 ceph-mon[34589]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:11:21.888 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:21 smithi183 ceph-mon[34589]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:11:21.888 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:21 smithi183 ceph-mon[34589]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:11:21.888 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:21.888 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:21 smithi183 ceph-mon[34589]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:11:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:21 smithi194 ceph-mon[34941]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T01:11:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:21 smithi194 ceph-mon[34941]: osdmap e60: 10 total, 9 up, 10 in 2024-09-05T01:11:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:21 smithi194 ceph-mon[34941]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-05T01:11:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:22.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:21 smithi194 ceph-mon[34941]: from='osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-05T01:11:22.342 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 -- 172.21.15.183:0/3232163859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58102690 msgr2=0x7f4e58108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/3232163859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58102690 0x7f4e58108f20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009ca0 tx=0x7f4e5002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 -- 172.21.15.183:0/3232163859 shutdown_connections 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/3232163859 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58102690 0x7f4e58108f20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/3232163859 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58101cd0 0x7f4e58102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/3232163859 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 0x7f4e58100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 -- 172.21.15.183:0/3232163859 >> 172.21.15.183:0/3232163859 conn(0x7f4e580fc280 msgr2=0x7f4e580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 -- 172.21.15.183:0/3232163859 shutdown_connections 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 -- 172.21.15.183:0/3232163859 wait complete. 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.328+0000 7f4e5f34b640 1 Processor -- start 2024-09-05T01:11:22.343 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 -- start start 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 0x7f4e58197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58102690 0x7f4e5819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810c5e0 con 0x7f4e58102690 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810c460 con 0x7f4e58101cd0 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5f34b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5810c760 con 0x7f4e58100ad0 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:41210/0 (socket says 172.21.15.183:41210) 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 -- 172.21.15.183:0/95381912 learned_addr learned my addr 172.21.15.183:0/95381912 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-09-05T01:11:22.344 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5d0e7640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 0x7f4e58197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5d8e8640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58102690 0x7f4e5819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 -- 172.21.15.183:0/95381912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 msgr2=0x7f4e58197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 0x7f4e58197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 -- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58102690 msgr2=0x7f4e5819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58102690 0x7f4e5819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 -- 172.21.15.183:0/95381912 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5819fa20 con 0x7f4e58101cd0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.329+0000 7f4e5c8e6640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.330+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800cd80 con 0x7f4e58101cd0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.330+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e480108e0 con 0x7f4e58101cd0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.330+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48018920 con 0x7f4e58101cd0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.330+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5819fd10 con 0x7f4e58101cd0 2024-09-05T01:11:22.345 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.330+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e581a01f0 con 0x7f4e58101cd0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.331+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e581a0420 con 0x7f4e58101cd0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.332+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4e48010a90 con 0x7f4e58101cd0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.333+0000 7f4e4e7fc640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 0x7f4e440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.333+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7f4e480ceed0 con 0x7f4e58101cd0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.333+0000 7f4e5d0e7640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 0x7f4e440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.333+0000 7f4e5d0e7640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 0x7f4e440a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e58101b30 tx=0x7f4e5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:11:22.346 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.336+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e4809c060 con 0x7f4e58101cd0 2024-09-05T01:11:22.517 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:22.516+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4e5806a6f0 con 0x7f4e440a2bb0 2024-09-05T01:11:22.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:22.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:22.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:22.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.121 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:23.121 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:11:23.121 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.121 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.122 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: osdmap e61: 10 total, 9 up, 10 in 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:23.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:24.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:11:24.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:11:24.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:24.117 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi129 2024-09-05T01:11:24.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-05T01:11:24.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] boot 2024-09-05T01:11:24.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:11:24.118 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi129 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] boot 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:11:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi129 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: osd.9 [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] boot 2024-09-05T01:11:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: osdmap e62: 10 total, 10 up, 10 in 2024-09-05T01:11:24.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T01:11:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:24 smithi183 ceph-mon[34589]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:24 smithi183 ceph-mon[34589]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:11:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:24 smithi129 ceph-mon[30979]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:24 smithi129 ceph-mon[30979]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:11:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:24 smithi194 ceph-mon[34941]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-05T01:11:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:24 smithi194 ceph-mon[34941]: osdmap e63: 10 total, 10 up, 10 in 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/4269253847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]': finished 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.183:0/2321322299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/4269253847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]': finished 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.183:0/2321322299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/4269253847' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]: dispatch 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2d90543e-16fb-4460-ad34-8e7d09c8c119"}]': finished 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: osdmap e64: 11 total, 10 up, 11 in 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.183:0/2321322299' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:27.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:26 smithi194 ceph-mon[34941]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:27.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:26 smithi183 ceph-mon[34589]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:26 smithi129 ceph-mon[30979]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:29.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:28 smithi183 ceph-mon[34589]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:28 smithi129 ceph-mon[30979]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:28 smithi194 ceph-mon[34941]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 304 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:31.146 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:30 smithi183 ceph-mon[34589]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:30 smithi129 ceph-mon[30979]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:30 smithi194 ceph-mon[34941]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:32.813 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:32 smithi183 ceph-mon[34589]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:32 smithi129 ceph-mon[30979]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:32 smithi194 ceph-mon[34941]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:34.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:33 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:11:34.053 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:33 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:33 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:11:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:33 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:34.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:33 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T01:11:34.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:33 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:35.057 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:34 smithi183 ceph-mon[34589]: Deploying daemon osd.10 on smithi183 2024-09-05T01:11:35.058 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:34 smithi183 ceph-mon[34589]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:34 smithi129 ceph-mon[30979]: Deploying daemon osd.10 on smithi183 2024-09-05T01:11:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:34 smithi129 ceph-mon[30979]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:34 smithi194 ceph-mon[34941]: Deploying daemon osd.10 on smithi183 2024-09-05T01:11:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:34 smithi194 ceph-mon[34941]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:37.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:36 smithi183 ceph-mon[34589]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:36 smithi129 ceph-mon[30979]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:36 smithi194 ceph-mon[34941]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:39.097 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:38 smithi183 ceph-mon[34589]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:38 smithi129 ceph-mon[30979]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:38 smithi194 ceph-mon[34941]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:40.091 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:39 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:40.091 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:39 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.091 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:39 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:39 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:39 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:39 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:39 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:40.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:40 smithi183 ceph-mon[34589]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:40 smithi129 ceph-mon[30979]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:40 smithi194 ceph-mon[34941]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:42 smithi183 ceph-mon[34589]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.227 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.227+0000 7f4e4e7fc640 1 -- 172.21.15.183:0/95381912 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f4e5806a6f0 con 0x7f4e440a2bb0 2024-09-05T01:11:43.228 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 msgr2=0x7f4e440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 0x7f4e440a5070 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e58101b30 tx=0x7f4e5400b390 comp rx=0 tx=0).stop 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 msgr2=0x7f4e58198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9f0 tx=0x7f4e4800eeb0 comp rx=0 tx=0).stop 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 shutdown_connections 2024-09-05T01:11:43.230 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4e440a2bb0 0x7f4e440a5070 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e58102690 0x7f4e5819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e58101cd0 0x7f4e58198330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 --2- 172.21.15.183:0/95381912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4e58100ad0 0x7f4e58197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 >> 172.21.15.183:0/95381912 conn(0x7f4e580fc280 msgr2=0x7f4e58069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.229+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 shutdown_connections 2024-09-05T01:11:43.231 INFO:teuthology.orchestra.run.smithi183.stderr:2024-09-05T01:11:43.230+0000 7f4e5f34b640 1 -- 172.21.15.183:0/95381912 wait complete. 2024-09-05T01:11:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:42 smithi129 ceph-mon[30979]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:42 smithi194 ceph-mon[34941]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:43.873 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.10.service 2024-09-05T01:11:43.875 INFO:tasks.cephadm:Deploying osd.11 on smithi194 with /dev/vg_nvme/lv_1... 2024-09-05T01:11:43.876 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.088 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.134 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:11:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:45.061 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:44 smithi183 ceph-mon[34589]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:44 smithi129 ceph-mon[30979]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:44 smithi194 ceph-mon[34941]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:45.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:45 smithi183 ceph-mon[34589]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:45.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:45 smithi183 ceph-mon[34589]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:45.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:11:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:11:45.603+0000 7f886eb91640 -1 osd.10 0 log_to_monitors true 2024-09-05T01:11:46.070 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:45 smithi194 ceph-mon[34941]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:46.071 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:45 smithi194 ceph-mon[34941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:45 smithi129 ceph-mon[30979]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:45 smithi129 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T01:11:47.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:47.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:47.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:47.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:47.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T01:11:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: osdmap e65: 11 total, 10 up, 11 in 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:11:47.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:11:47.371 INFO:teuthology.orchestra.run.smithi194.stdout: 2024-09-05T01:11:47.396 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi194:vg_nvme/lv_1 2024-09-05T01:11:47.649 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:11:47.829 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:11:47.485+0000 7f886ab14640 -1 osd.10 0 waiting for initial osdmap 2024-09-05T01:11:47.830 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:11:47.495+0000 7f8862934640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:11:48.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi183 2024-09-05T01:11:48.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-05T01:11:48.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:11:48.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:11:48.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:47 smithi194 ceph-mon[34941]: from='osd.10 ' entity='osd.10' 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 -- 172.21.15.194:0/1545973642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0101c00 msgr2=0x7f6fa0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 --2- 172.21.15.194:0/1545973642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0101c00 0x7f6fa0102000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6f90009900 tx=0x7f6f9002a410 comp rx=0 tx=0).stop 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 -- 172.21.15.194:0/1545973642 shutdown_connections 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 --2- 172.21.15.194:0/1545973642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 --2- 172.21.15.194:0/1545973642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0102e00 0x7f6fa0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 --2- 172.21.15.194:0/1545973642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0101c00 0x7f6fa0102000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 -- 172.21.15.194:0/1545973642 >> 172.21.15.194:0/1545973642 conn(0x7f6fa00fd390 msgr2=0x7f6fa00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 -- 172.21.15.194:0/1545973642 shutdown_connections 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.128+0000 7f6fa619c640 1 -- 172.21.15.194:0/1545973642 wait complete. 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 Processor -- start 2024-09-05T01:11:48.147 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 -- start start 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0101c00 0x7f6fa0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010d790 con 0x7f6fa0101c00 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010d610 con 0x7f6fa01037c0 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6fa619c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010d910 con 0x7f6fa0102e00 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:49064/0 (socket says 172.21.15.194:49064) 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9effd640 1 -- 172.21.15.194:0/622693907 learned_addr learned my addr 172.21.15.194:0/622693907 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9ffff640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:48.148 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9effd640 1 -- 172.21.15.194:0/622693907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 msgr2=0x7f6fa0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.129+0000 7f6f9f7fe640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0101c00 0x7f6fa0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9effd640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9effd640 1 -- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0101c00 msgr2=0x7f6fa0075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9effd640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0101c00 0x7f6fa0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9effd640 1 -- 172.21.15.194:0/622693907 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa00753e0 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9ffff640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9effd640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400d950 tx=0x7f6f9400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94017030 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f94004020 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f940047a0 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa01a1fd0 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.130+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa01a24d0 con 0x7f6fa0102e00 2024-09-05T01:11:48.149 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.131+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa01090a0 con 0x7f6fa0102e00 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.132+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6f940041d0 con 0x7f6fa0102e00 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.133+0000 7f6f9cff9640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 0x7f6f880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.133+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6723+0+0 (secure 0 0 0) 0x7f6f940ce550 con 0x7f6fa0102e00 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.133+0000 7f6f9f7fe640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 0x7f6f880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.133+0000 7f6f9f7fe640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 0x7f6f880a50c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f9002a930 tx=0x7f6f90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:11:48.150 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.137+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f9409bae0 con 0x7f6fa0102e00 2024-09-05T01:11:48.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi183 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:47 smithi183 ceph-mon[34589]: from='osd.10 ' entity='osd.10' 2024-09-05T01:11:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi183 2024-09-05T01:11:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 4272M 2024-09-05T01:11:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-09-05T01:11:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: osdmap e66: 11 total, 10 up, 11 in 2024-09-05T01:11:48.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:48.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:47 smithi129 ceph-mon[30979]: from='osd.10 ' entity='osd.10' 2024-09-05T01:11:48.314 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:11:48.312+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6fa0062630 con 0x7f6f880a2c00 2024-09-05T01:11:49.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:11:49.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:11:49.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] boot 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:49.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] boot 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:49.144 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: osd.10 [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] boot 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: osdmap e67: 11 total, 11 up, 11 in 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: osdmap e68: 11 total, 11 up, 11 in 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T01:11:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:11:50.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:49 smithi183 ceph-mon[34589]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:50.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:49 smithi183 ceph-mon[34589]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:11:50.176 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:49 smithi194 ceph-mon[34941]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:50.176 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:49 smithi194 ceph-mon[34941]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:11:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:49 smithi129 ceph-mon[30979]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi194:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:11:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:49 smithi129 ceph-mon[30979]: osdmap e69: 11 total, 11 up, 11 in 2024-09-05T01:11:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:50 smithi183 ceph-mon[34589]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:50 smithi183 ceph-mon[34589]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T01:11:51.185 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:50 smithi194 ceph-mon[34941]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:51.186 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:50 smithi194 ceph-mon[34941]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T01:11:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:50 smithi129 ceph-mon[30979]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:50 smithi129 ceph-mon[30979]: osdmap e70: 11 total, 11 up, 11 in 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/3837448484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]': finished 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:11:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.194:0/2431304448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/3837448484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]': finished 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:11:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.194:0/2431304448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/3837448484' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]: dispatch 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa637f9f-a4b5-41aa-ab8f-66306b7c6565"}]': finished 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: osdmap e71: 12 total, 11 up, 12 in 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:11:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.194:0/2431304448' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T01:11:53.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:52 smithi183 ceph-mon[34589]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:52 smithi129 ceph-mon[30979]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:52 smithi194 ceph-mon[34941]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:11:55.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:54 smithi183 ceph-mon[34589]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-05T01:11:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:54 smithi129 ceph-mon[30979]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-05T01:11:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:54 smithi194 ceph-mon[34941]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-05T01:11:57.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:56 smithi183 ceph-mon[34589]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-05T01:11:57.182 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:56 smithi194 ceph-mon[34941]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-05T01:11:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:56 smithi129 ceph-mon[30979]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-05T01:11:58.901 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:58 smithi194 ceph-mon[34941]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:11:59.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:58 smithi183 ceph-mon[34589]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:11:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:58 smithi129 ceph-mon[30979]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T01:12:00.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:59 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:12:00.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:11:59 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:00.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:59 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:12:00.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:11:59 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:00.243 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:59 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T01:12:00.243 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:11:59 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:00 smithi183 ceph-mon[34589]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:12:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:00 smithi183 ceph-mon[34589]: Deploying daemon osd.11 on smithi194 2024-09-05T01:12:01.239 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:00 smithi194 ceph-mon[34941]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:12:01.239 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:00 smithi194 ceph-mon[34941]: Deploying daemon osd.11 on smithi194 2024-09-05T01:12:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:00 smithi129 ceph-mon[30979]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T01:12:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:00 smithi129 ceph-mon[30979]: Deploying daemon osd.11 on smithi194 2024-09-05T01:12:03.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:02 smithi194 ceph-mon[34941]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T01:12:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:02 smithi183 ceph-mon[34589]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T01:12:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:02 smithi129 ceph-mon[30979]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T01:12:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:04 smithi183 ceph-mon[34589]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:12:05.178 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:04 smithi194 ceph-mon[34941]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:12:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:04 smithi129 ceph-mon[30979]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T01:12:06.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:05 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:06.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:05 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:06.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:05 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:06.196 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:06.196 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:06.196 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:06.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:06 smithi183 ceph-mon[34589]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:06 smithi129 ceph-mon[30979]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:06 smithi194 ceph-mon[34941]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:09.162 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:09.462 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.462+0000 7f6f9cff9640 1 -- 172.21.15.194:0/622693907 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6fa0062630 con 0x7f6f880a2c00 2024-09-05T01:12:09.463 INFO:teuthology.orchestra.run.smithi194.stdout:Created osd(s) 11 on host 'smithi194' 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 msgr2=0x7f6f880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 0x7f6f880a50c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6f9002a930 tx=0x7f6f90000bc0 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 msgr2=0x7f6fa00745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6f9400d950 tx=0x7f6f9400de10 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 shutdown_connections 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6f880a2c00 0x7f6f880a50c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6fa01037c0 0x7f6fa0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6fa0102e00 0x7f6fa00745e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 --2- 172.21.15.194:0/622693907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6fa0101c00 0x7f6fa0075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 >> 172.21.15.194:0/622693907 conn(0x7f6fa00fd390 msgr2=0x7f6fa0106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:09.465 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 shutdown_connections 2024-09-05T01:12:09.466 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:09.464+0000 7f6fa619c640 1 -- 172.21.15.194:0/622693907 wait complete. 2024-09-05T01:12:10.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:10.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.176 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:10.176 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.176 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.203 DEBUG:teuthology.orchestra.run.smithi194:osd.11> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.11.service 2024-09-05T01:12:10.205 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-05T01:12:10.206 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:12:10.234 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:10.234 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.234 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:10.467 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 -- 172.21.15.129:0/192094612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4101d10 msgr2=0x7f90f4102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 --2- 172.21.15.129:0/192094612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4101d10 0x7f90f4102190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e402a410 comp rx=0 tx=0).stop 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 -- 172.21.15.129:0/192094612 shutdown_connections 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 --2- 172.21.15.129:0/192094612 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f41026d0 0x7f90f4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 --2- 172.21.15.129:0/192094612 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4101d10 0x7f90f4102190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:10.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 --2- 172.21.15.129:0/192094612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f4100b10 0x7f90f4100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 -- 172.21.15.129:0/192094612 >> 172.21.15.129:0/192094612 conn(0x7f90f40fc2a0 msgr2=0x7f90f40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 -- 172.21.15.129:0/192094612 shutdown_connections 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.980+0000 7f90fc670640 1 -- 172.21.15.129:0/192094612 wait complete. 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 Processor -- start 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 -- start start 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f4101d10 0x7f90f4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 0x7f90f419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90f410d770 con 0x7f90f4100b10 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f90f410d5f0 con 0x7f90f41026d0 2024-09-05T01:12:10.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fc670640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f90f410d8f0 con 0x7f90f4101d10 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38874/0 (socket says 172.21.15.129:38874) 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 -- 172.21.15.129:0/1484916351 learned_addr learned my addr 172.21.15.129:0/1484916351 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90f9c0b640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f4101d10 0x7f90f4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fac0d640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 0x7f90f419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 -- 172.21.15.129:0/1484916351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f4101d10 msgr2=0x7f90f4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f4101d10 0x7f90f4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.981+0000 7f90fa40c640 1 -- 172.21.15.129:0/1484916351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 msgr2=0x7f90f419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fa40c640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 0x7f90f419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fa40c640 1 -- 172.21.15.129:0/1484916351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f419fa50 con 0x7f90f4100b10 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fac0d640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 0x7f90f419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:10.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fa40c640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9c0 tx=0x7f90f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:10.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000cd80 con 0x7f90f4100b10 2024-09-05T01:12:10.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f419fd40 con 0x7f90f4100b10 2024-09-05T01:12:10.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90f00035f0 con 0x7f90f4100b10 2024-09-05T01:12:10.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0010690 con 0x7f90f4100b10 2024-09-05T01:12:10.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.982+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f41a01a0 con 0x7f90f4100b10 2024-09-05T01:12:10.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:10 smithi129 ceph-mon[30979]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.983+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f4107f60 con 0x7f90f4100b10 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.985+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f90f00037a0 con 0x7f90f4100b10 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.985+0000 7f90eb7fe640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 0x7f90e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.985+0000 7f90f9c0b640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 0x7f90e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.985+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f90f00cee60 con 0x7f90f4100b10 2024-09-05T01:12:10.988 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.986+0000 7f90f9c0b640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 0x7f90e00a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90e4006010 tx=0x7f90e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:10.990 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:10.989+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90f009c380 con 0x7f90f4100b10 2024-09-05T01:12:11.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.154+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f90f402d990 con 0x7f90f4100b10 2024-09-05T01:12:11.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.154+0000 7f90eb7fe640 1 -- 172.21.15.129:0/1484916351 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f90f009bc80 con 0x7f90f4100b10 2024-09-05T01:12:11.155 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 msgr2=0x7f90e00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 0x7f90e00a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f90e4006010 tx=0x7f90e4000bc0 comp rx=0 tx=0).stop 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 msgr2=0x7f90f4197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9c0 tx=0x7f90f000ee80 comp rx=0 tx=0).stop 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 shutdown_connections 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f90e00a2bb0 0x7f90e00a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f90f41026d0 0x7f90f419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f90f4101d10 0x7f90f4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:11.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 --2- 172.21.15.129:0/1484916351 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90f4100b10 0x7f90f4197e20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:11.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 >> 172.21.15.129:0/1484916351 conn(0x7f90f40fc2a0 msgr2=0x7f90f4105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:11.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 shutdown_connections 2024-09-05T01:12:11.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:11.156+0000 7f90fc670640 1 -- 172.21.15.129:0/1484916351 wait complete. 2024-09-05T01:12:11.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:10 smithi183 ceph-mon[34589]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:10 smithi194 ceph-mon[34941]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:11.579 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:12:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:12:11.257+0000 7f01a5812640 -1 osd.11 0 log_to_monitors true 2024-09-05T01:12:11.983 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1725498707,"num_in_osds":12,"osd_in_since":1725498710,"num_remapped_pgs":0} 2024-09-05T01:12:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:11 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1484916351' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:11 smithi183 ceph-mon[34589]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:11 smithi183 ceph-mon[34589]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:11 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1484916351' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:11 smithi129 ceph-mon[30979]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:11 smithi129 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:11 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1484916351' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:11 smithi194 ceph-mon[34941]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:11 smithi194 ceph-mon[34941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T01:12:12.985 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:12 smithi183 ceph-mon[34589]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.234 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:12 smithi129 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: osdmap e72: 12 total, 11 up, 12 in 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: from='osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:12 smithi194 ceph-mon[34941]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]: dispatch 2024-09-05T01:12:13.750 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:12:13.579+0000 7f01a1fa9640 -1 osd.11 0 waiting for initial osdmap 2024-09-05T01:12:13.750 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:12:13.589+0000 7f01995b5640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T01:12:13.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 -- 172.21.15.129:0/2372916747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580101cd0 msgr2=0x7fe580102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2372916747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580101cd0 0x7fe580102150 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe568009900 tx=0x7fe56802a410 comp rx=0 tx=0).stop 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 -- 172.21.15.129:0/2372916747 shutdown_connections 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2372916747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580102690 0x7fe580108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2372916747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580101cd0 0x7fe580102150 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2372916747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe580100ad0 0x7fe580100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 -- 172.21.15.129:0/2372916747 >> 172.21.15.129:0/2372916747 conn(0x7fe5800fc280 msgr2=0x7fe5800fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 -- 172.21.15.129:0/2372916747 shutdown_connections 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.729+0000 7fe5858ee640 1 -- 172.21.15.129:0/2372916747 wait complete. 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 Processor -- start 2024-09-05T01:12:13.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 -- start start 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 0x7fe5801030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe580102690 0x7fe5801035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe58010c700 con 0x7fe580100ad0 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe58010c580 con 0x7fe580102690 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe5858ee640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe58010c880 con 0x7fe580101cd0 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe57effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe57effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41158/0 (socket says 172.21.15.129:41158) 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe57effd640 1 -- 172.21.15.129:0/2440623457 learned_addr learned my addr 172.21.15.129:0/2440623457 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.730+0000 7fe577fff640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 0x7fe5801030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 -- 172.21.15.129:0/2440623457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 msgr2=0x7fe5801030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 0x7fe5801030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 -- 172.21.15.129:0/2440623457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe580102690 msgr2=0x7fe5801035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe580102690 0x7fe5801035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 -- 172.21.15.129:0/2440623457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe580103eb0 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe577fff640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 0x7fe5801030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe57effd640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b450 tx=0x7fe57000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570015020 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe570011d40 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5801a2180 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57000f340 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.731+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5801a2760 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.732+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5801a29f0 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.733+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe570018020 con 0x7fe580100ad0 2024-09-05T01:12:13.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.734+0000 7fe5848ec640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 0x7fe56c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:13.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.734+0000 7fe577fff640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 0x7fe56c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:13.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.734+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7fe5700ce550 con 0x7fe580100ad0 2024-09-05T01:12:13.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.734+0000 7fe577fff640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 0x7fe56c0a50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe5801047e0 tx=0x7fe568031d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:13.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.738+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe57001f020 con 0x7fe580100ad0 2024-09-05T01:12:13.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.907+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fe5801a2bc0 con 0x7fe580100ad0 2024-09-05T01:12:13.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.907+0000 7fe5848ec640 1 -- 172.21.15.129:0/2440623457 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7fe57000d040 con 0x7fe580100ad0 2024-09-05T01:12:13.908 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:13.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:12:13.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:12:13.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:13.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:13.908 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:13.909 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:13 smithi129 ceph-mon[30979]: from='osd.11 ' entity='osd.11' 2024-09-05T01:12:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 msgr2=0x7fe56c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 0x7fe56c0a50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe5801047e0 tx=0x7fe568031d10 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 msgr2=0x7fe580104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe57000b450 tx=0x7fe57000b910 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 shutdown_connections 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe56c0a2c00 0x7fe56c0a50c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe580102690 0x7fe5801035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe580101cd0 0x7fe5801030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 --2- 172.21.15.129:0/2440623457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe580100ad0 0x7fe580104a60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 >> 172.21.15.129:0/2440623457 conn(0x7fe5800fc280 msgr2=0x7fe580069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 shutdown_connections 2024-09-05T01:12:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:13.909+0000 7fe5858ee640 1 -- 172.21.15.129:0/2440623457 wait complete. 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:14.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:14.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:13 smithi183 ceph-mon[34589]: from='osd.11 ' entity='osd.11' 2024-09-05T01:12:14.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi194", "root=default"]}]': finished 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: osdmap e73: 12 total, 11 up, 12 in 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:14.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:14.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:13 smithi194 ceph-mon[34941]: from='osd.11 ' entity='osd.11' 2024-09-05T01:12:14.727 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1725498707,"num_in_osds":12,"osd_in_since":1725498710,"num_remapped_pgs":0} 2024-09-05T01:12:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: purged_snaps scrub starts 2024-09-05T01:12:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: purged_snaps scrub ok 2024-09-05T01:12:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: Detected new or changed devices on smithi194 2024-09-05T01:12:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 4272M 2024-09-05T01:12:14.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:14.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:14.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2440623457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:14.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] boot 2024-09-05T01:12:14.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:12:14.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: purged_snaps scrub starts 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: purged_snaps scrub ok 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: Detected new or changed devices on smithi194 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 4272M 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2440623457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] boot 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:12:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: purged_snaps scrub starts 2024-09-05T01:12:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: purged_snaps scrub ok 2024-09-05T01:12:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: Detected new or changed devices on smithi194 2024-09-05T01:12:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 4272M 2024-09-05T01:12:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-09-05T01:12:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2440623457' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: osd.11 [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] boot 2024-09-05T01:12:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: osdmap e74: 12 total, 12 up, 12 in 2024-09-05T01:12:15.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T01:12:15.728 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T01:12:15.978 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 -- 172.21.15.129:0/856490404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 msgr2=0x7f5178100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 --2- 172.21.15.129:0/856490404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178100f10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5168009900 tx=0x7f516802a410 comp rx=0 tx=0).stop 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 -- 172.21.15.129:0/856490404 shutdown_connections 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 --2- 172.21.15.129:0/856490404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f5178108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 --2- 172.21.15.129:0/856490404 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f5178102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 --2- 172.21.15.129:0/856490404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178100f10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 -- 172.21.15.129:0/856490404 >> 172.21.15.129:0/856490404 conn(0x7f51780fc2a0 msgr2=0x7f51780fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 -- 172.21.15.129:0/856490404 shutdown_connections 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.431+0000 7f517da8e640 1 -- 172.21.15.129:0/856490404 wait complete. 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 Processor -- start 2024-09-05T01:12:16.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 -- start start 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f51781983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f517819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d7d0 con 0x7f5178100b10 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d650 con 0x7f51781026d0 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f517da8e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f517810d950 con 0x7f5178101d10 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41162/0 (socket says 172.21.15.129:41162) 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 -- 172.21.15.129:0/295799222 learned_addr learned my addr 172.21.15.129:0/295799222 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f5176ffd640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f51781983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f5177fff640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f517819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 -- 172.21.15.129:0/295799222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 msgr2=0x7f51781983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f51781983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 -- 172.21.15.129:0/295799222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 msgr2=0x7f517819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f517819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f51777fe640 1 -- 172.21.15.129:0/295799222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f517819fab0 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f5176ffd640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f51781983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.432+0000 7f5177fff640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f517819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f51777fe640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51680314e0 tx=0x7f51680042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516803f030 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f517819fd40 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5168003cb0 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5168035070 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.433+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51781a01c0 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.434+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5178107f50 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.435+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5168042030 con 0x7f5178100b10 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.436+0000 7f5174ff9640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 0x7f51600a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:16.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.436+0000 7f5176ffd640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 0x7f51600a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:16.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.436+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f51680ef6f0 con 0x7f5178100b10 2024-09-05T01:12:16.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.436+0000 7f5176ffd640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 0x7f51600a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f516c0058c0 tx=0x7f516c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:16.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.439+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51680bca50 con 0x7f5178100b10 2024-09-05T01:12:16.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.603+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f517802d990 con 0x7f5178100b10 2024-09-05T01:12:16.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.604+0000 7f5174ff9640 1 -- 172.21.15.129:0/295799222 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f51680bc350 con 0x7f5178100b10 2024-09-05T01:12:16.604 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:16.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 msgr2=0x7f51600a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:16.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 0x7f51600a5070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f516c0058c0 tx=0x7f516c005850 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 msgr2=0x7f5178197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f51680314e0 tx=0x7f51680042e0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 shutdown_connections 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f51600a2bb0 0x7f51600a5070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51781026d0 0x7f517819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5178101d10 0x7f51781983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 --2- 172.21.15.129:0/295799222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5178100b10 0x7f5178197e80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 >> 172.21.15.129:0/295799222 conn(0x7f51780fc2a0 msgr2=0x7f5178105120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 shutdown_connections 2024-09-05T01:12:16.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:16.606+0000 7f517da8e640 1 -- 172.21.15.129:0/295799222 wait complete. 2024-09-05T01:12:17.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:16 smithi183 ceph-mon[34589]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:16 smithi183 ceph-mon[34589]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:12:17.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:16 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/295799222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:16 smithi129 ceph-mon[30979]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:16 smithi129 ceph-mon[30979]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:12:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:16 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/295799222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:16 smithi194 ceph-mon[34941]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:16 smithi194 ceph-mon[34941]: osdmap e75: 12 total, 12 up, 12 in 2024-09-05T01:12:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:16 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/295799222' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T01:12:17.448 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1725498734,"num_in_osds":12,"osd_in_since":1725498710,"num_remapped_pgs":0} 2024-09-05T01:12:17.452 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:12:17.698 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:18.103 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:17 smithi129 ceph-mon[30979]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 -- 172.21.15.129:0/3895948988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0401002f0 msgr2=0x7fe0401006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 --2- 172.21.15.129:0/3895948988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0401002f0 0x7fe0401006d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009930 tx=0x7fe03c02a420 comp rx=0 tx=0).stop 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 -- 172.21.15.129:0/3895948988 shutdown_connections 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 --2- 172.21.15.129:0/3895948988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040106f50 0x7fe04010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 --2- 172.21.15.129:0/3895948988 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe040100ca0 0x7fe040106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 --2- 172.21.15.129:0/3895948988 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0401002f0 0x7fe0401006d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 -- 172.21.15.129:0/3895948988 >> 172.21.15.129:0/3895948988 conn(0x7fe040077210 msgr2=0x7fe0400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 -- 172.21.15.129:0/3895948988 shutdown_connections 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.180+0000 7fe048511640 1 -- 172.21.15.129:0/3895948988 wait complete. 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 Processor -- start 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 -- start start 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0401002f0 0x7fe0401983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 0x7fe040198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe040111e50 con 0x7fe040106f50 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe040111cd0 con 0x7fe040100ca0 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.181+0000 7fe048511640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe040111fd0 con 0x7fe0401002f0 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:18.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41186/0 (socket says 172.21.15.129:41186) 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 -- 172.21.15.129:0/1234925081 learned_addr learned my addr 172.21.15.129:0/1234925081 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe0462ad640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0401002f0 0x7fe0401983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe045aac640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 0x7fe040198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 -- 172.21.15.129:0/1234925081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0401002f0 msgr2=0x7fe0401983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0401002f0 0x7fe0401983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 -- 172.21.15.129:0/1234925081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 msgr2=0x7fe040198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 0x7fe040198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 -- 172.21.15.129:0/1234925081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04019d3a0 con 0x7fe040106f50 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe045aac640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 0x7fe040198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe046aae640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe03800cad0 tx=0x7fe03800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:18.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03800ae20 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04019d690 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe038005020 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03800f6b0 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.182+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0401a4f70 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.183+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0401006d0 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.184+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe0380051d0 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.185+0000 7fe0377fe640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 0x7fe02c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.185+0000 7fe0462ad640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 0x7fe02c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.185+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe0380cebc0 con 0x7fe040106f50 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.186+0000 7fe0462ad640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 0x7fe02c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006010 tx=0x7fe03c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:18.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.190+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe038014030 con 0x7fe040106f50 2024-09-05T01:12:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:17 smithi194 ceph-mon[34941]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:18.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.356+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe040109bb0 con 0x7fe040106f50 2024-09-05T01:12:18.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.357+0000 7fe0377fe640 1 -- 172.21.15.129:0/1234925081 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18515 (secure 0 0 0) 0x7fe03809bfc0 con 0x7fe040106f50 2024-09-05T01:12:18.358 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:18.358 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":75,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:12:15.888572+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:12:18.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 msgr2=0x7fe02c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 0x7fe02c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006010 tx=0x7fe03c000bc0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 msgr2=0x7fe04019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe03800cad0 tx=0x7fe03800cf90 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 shutdown_connections 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe02c0a2b80 0x7fe02c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe040106f50 0x7fe04019ccc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe040100ca0 0x7fe040198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 --2- 172.21.15.129:0/1234925081 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0401002f0 0x7fe0401983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 >> 172.21.15.129:0/1234925081 conn(0x7fe040077210 msgr2=0x7fe040105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 shutdown_connections 2024-09-05T01:12:18.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:18.360+0000 7fe048511640 1 -- 172.21.15.129:0/1234925081 wait complete. 2024-09-05T01:12:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:17 smithi183 ceph-mon[34589]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:19.133 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:18 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1234925081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:12:19.163 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T01:08:27.962073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T01:12:19.163 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-05T01:12:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:18 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1234925081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:12:19.406 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:18 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1234925081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 -- 172.21.15.129:0/1472394142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0106f10 msgr2=0x7fd8c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 --2- 172.21.15.129:0/1472394142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0106f10 0x7fd8c010ea90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8009ca0 tx=0x7fd8b802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 -- 172.21.15.129:0/1472394142 shutdown_connections 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 --2- 172.21.15.129:0/1472394142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0106f10 0x7fd8c010ea90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 --2- 172.21.15.129:0/1472394142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0102790 0x7fd8c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 --2- 172.21.15.129:0/1472394142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0101de0 0x7fd8c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.861+0000 7fd8c7980640 1 -- 172.21.15.129:0/1472394142 >> 172.21.15.129:0/1472394142 conn(0x7fd8c00fd8c0 msgr2=0x7fd8c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- 172.21.15.129:0/1472394142 shutdown_connections 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- 172.21.15.129:0/1472394142 wait complete. 2024-09-05T01:12:19.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 Processor -- start 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- start start 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0101de0 0x7fd8c0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0106f10 0x7fd8c019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0111e40 con 0x7fd8c0102790 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0111cc0 con 0x7fd8c0106f10 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.862+0000 7fd8c7980640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0111fc0 con 0x7fd8c0101de0 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41206/0 (socket says 172.21.15.129:41206) 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 -- 172.21.15.129:0/3334567219 learned_addr learned my addr 172.21.15.129:0/3334567219 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:19.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c5f1d640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0106f10 0x7fd8c019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c571c640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0101de0 0x7fd8c0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 -- 172.21.15.129:0/3334567219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0101de0 msgr2=0x7fd8c0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0101de0 0x7fd8c0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 -- 172.21.15.129:0/3334567219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0106f10 msgr2=0x7fd8c019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0106f10 0x7fd8c019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 -- 172.21.15.129:0/3334567219 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c019d3b0 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c4f1b640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b4a0 tx=0x7fd8b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0014070 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c019d6a0 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8b0010ac0 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0017c30 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.863+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c01a4f80 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.864+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c01021c0 con 0x7fd8c0102790 2024-09-05T01:12:19.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.865+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd8b0017070 con 0x7fd8c0102790 2024-09-05T01:12:19.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.866+0000 7fd8b67fc640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 0x7fd8ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:19.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.866+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd8b00ce000 con 0x7fd8c0102790 2024-09-05T01:12:19.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.866+0000 7fd8c571c640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 0x7fd8ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:19.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.866+0000 7fd8c571c640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 0x7fd8ac0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0058c0 tx=0x7fd8bc00a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:19.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:19.870+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8b009b400 con 0x7fd8c0102790 2024-09-05T01:12:20.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.038+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8c002d7c0 con 0x7fd8c0102790 2024-09-05T01:12:20.039 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:19 smithi129 ceph-mon[30979]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:20.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.039+0000 7fd8b67fc640 1 -- 172.21.15.129:0/3334567219 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fd8b009ad00 con 0x7fd8c0102790 2024-09-05T01:12:20.040 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-09-05T01:12:20.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 msgr2=0x7fd8ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 0x7fd8ac0a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0058c0 tx=0x7fd8bc00a4c0 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 msgr2=0x7fd8c0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000b4a0 tx=0x7fd8b000b960 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 shutdown_connections 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8ac0a2b80 0x7fd8ac0a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8c0106f10 0x7fd8c019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c0102790 0x7fd8c0198940 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 --2- 172.21.15.129:0/3334567219 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8c0101de0 0x7fd8c0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 >> 172.21.15.129:0/3334567219 conn(0x7fd8c00fd8c0 msgr2=0x7fd8c00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 shutdown_connections 2024-09-05T01:12:20.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:20.041+0000 7fd8c7980640 1 -- 172.21.15.129:0/3334567219 wait complete. 2024-09-05T01:12:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:19 smithi194 ceph-mon[34941]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:20.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:19 smithi183 ceph-mon[34589]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.a on smithi129 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.d on smithi129 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.g on smithi129 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.j on smithi129 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-09-05T01:12:20.722 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.c on smithi194 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.f on smithi194 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.i on smithi194 2024-09-05T01:12:20.723 INFO:tasks.cephadm:Adding mds.l on smithi194 2024-09-05T01:12:20.723 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph orch apply mds all '12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l' 2024-09-05T01:12:20.982 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:12:21.103 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:20 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3334567219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:12:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:20 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3334567219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:12:21.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:20 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3334567219' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 -- 172.21.15.194:0/1892747848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 msgr2=0x7f05e4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/1892747848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4103280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009900 tx=0x7f05e002a410 comp rx=0 tx=0).stop 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 -- 172.21.15.194:0/1892747848 shutdown_connections 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/1892747848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e41037c0 0x7f05e410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/1892747848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4103280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/1892747848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e4101c00 0x7f05e4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 -- 172.21.15.194:0/1892747848 >> 172.21.15.194:0/1892747848 conn(0x7f05e40fd390 msgr2=0x7f05e40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:21.488 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 -- 172.21.15.194:0/1892747848 shutdown_connections 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.470+0000 7f05ec2e6640 1 -- 172.21.15.194:0/1892747848 wait complete. 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 Processor -- start 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 -- start start 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 0x7f05e4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e41037c0 0x7f05e419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f05e410d710 con 0x7f05e4101c00 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f05e410d590 con 0x7f05e41037c0 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ec2e6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f05e410d890 con 0x7f05e4102e00 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.194:48798/0 (socket says 172.21.15.194:48798) 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 -- 172.21.15.194:0/67117267 learned_addr learned my addr 172.21.15.194:0/67117267 (peer_addr_for_me v2:172.21.15.194:0/0) 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ea883640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e41037c0 0x7f05e419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:21.489 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ea082640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 0x7f05e4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 -- 172.21.15.194:0/67117267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e41037c0 msgr2=0x7f05e419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e41037c0 0x7f05e419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 -- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 msgr2=0x7f05e4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 0x7f05e4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 -- 172.21.15.194:0/67117267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e419fa50 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05ea082640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 0x7f05e4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.471+0000 7f05e9881640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05e0004770 tx=0x7f05e00042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.472+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e003f030 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.472+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e419fce0 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.472+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05e0003cb0 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.472+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0035070 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.472+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e41a01c0 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.473+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e4109140 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.474+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f05e0042030 con 0x7f05e4102e00 2024-09-05T01:12:21.490 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.475+0000 7f05db7fe640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 0x7f05d00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:21.491 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.475+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f05e00ef210 con 0x7f05e4102e00 2024-09-05T01:12:21.491 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.475+0000 7f05ea082640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 0x7f05d00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:21.491 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.475+0000 7f05ea082640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 0x7f05d00a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005d00 tx=0x7f05d4005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:21.491 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.479+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05e00bc5b0 con 0x7f05e4102e00 2024-09-05T01:12:21.653 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.653+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}) -- 0x7f05e40628f0 con 0x7f05d00a2bb0 2024-09-05T01:12:21.660 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.659+0000 7f05db7fe640 1 -- 172.21.15.194:0/67117267 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f05e40628f0 con 0x7f05d00a2bb0 2024-09-05T01:12:21.660 INFO:teuthology.orchestra.run.smithi194.stdout:Scheduled mds.all update... 2024-09-05T01:12:21.662 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 msgr2=0x7f05d00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:21.662 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 0x7f05d00a5070 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005d00 tx=0x7f05d4005ef0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.662 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 msgr2=0x7f05e4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:21.662 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f05e0004770 tx=0x7f05e00042f0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 shutdown_connections 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05d00a2bb0 0x7f05d00a5070 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05e41037c0 0x7f05e419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f05e4102e00 0x7f05e4198360 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 --2- 172.21.15.194:0/67117267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f05e4101c00 0x7f05e4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.661+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 >> 172.21.15.194:0/67117267 conn(0x7f05e40fd390 msgr2=0x7f05e4106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.662+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 shutdown_connections 2024-09-05T01:12:21.663 INFO:teuthology.orchestra.run.smithi194.stderr:2024-09-05T01:12:21.662+0000 7f05ec2e6640 1 -- 172.21.15.194:0/67117267 wait complete. 2024-09-05T01:12:22.544 DEBUG:teuthology.orchestra.run.smithi129:mds.a> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.a.service 2024-09-05T01:12:22.547 DEBUG:teuthology.orchestra.run.smithi129:mds.d> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.d.service 2024-09-05T01:12:22.591 DEBUG:teuthology.orchestra.run.smithi129:mds.g> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.g.service 2024-09-05T01:12:22.635 DEBUG:teuthology.orchestra.run.smithi129:mds.j> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.j.service 2024-09-05T01:12:22.677 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.b.service 2024-09-05T01:12:22.680 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.e.service 2024-09-05T01:12:22.727 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.h.service 2024-09-05T01:12:22.770 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.k.service 2024-09-05T01:12:22.790 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:12:22.790 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-09-05T01:12:22.790 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:22.790 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:22.790 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:22.791 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:22.814 DEBUG:teuthology.orchestra.run.smithi194:mds.c> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.c.service 2024-09-05T01:12:22.817 DEBUG:teuthology.orchestra.run.smithi194:mds.f> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.f.service 2024-09-05T01:12:22.860 DEBUG:teuthology.orchestra.run.smithi194:mds.i> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.i.service 2024-09-05T01:12:22.903 DEBUG:teuthology.orchestra.run.smithi194:mds.l> sudo journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.l.service 2024-09-05T01:12:22.947 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-05T01:12:22.947 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:22.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:12:22.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-09-05T01:12:22.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:22.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:22.970 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:12:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: Saving service mds.all spec with placement smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-09-05T01:12:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:23.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:23.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:23.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:23.189 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:23.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 -- 172.21.15.129:0/1633995853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188106f10 msgr2=0x7f618810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:23.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 --2- 172.21.15.129:0/1633995853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188106f10 0x7f618810ea90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6180009ca0 tx=0x7f618002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 -- 172.21.15.129:0/1633995853 shutdown_connections 2024-09-05T01:12:23.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 --2- 172.21.15.129:0/1633995853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188106f10 0x7f618810ea90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 --2- 172.21.15.129:0/1633995853 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188102790 0x7f61881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 --2- 172.21.15.129:0/1633995853 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188101de0 0x7f61881021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 -- 172.21.15.129:0/1633995853 >> 172.21.15.129:0/1633995853 conn(0x7f61880fd8c0 msgr2=0x7f61880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 -- 172.21.15.129:0/1633995853 shutdown_connections 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.658+0000 7f618effd640 1 -- 172.21.15.129:0/1633995853 wait complete. 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 Processor -- start 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 -- start start 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 0x7f61881982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188106f10 0x7f618819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:23.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6188111e20 con 0x7f6188102790 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6188111ca0 con 0x7f6188101de0 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f618effd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6188111fa0 con 0x7f6188106f10 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f6187fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f6187fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40710/0 (socket says 172.21.15.129:40710) 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.659+0000 7f6187fff640 1 -- 172.21.15.129:0/940128010 learned_addr learned my addr 172.21.15.129:0/940128010 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f618d59a640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188106f10 0x7f618819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f618cd99640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 0x7f61881982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 -- 172.21.15.129:0/940128010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188106f10 msgr2=0x7f618819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188106f10 0x7f618819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 -- 172.21.15.129:0/940128010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 msgr2=0x7f61881982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 0x7f61881982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 -- 172.21.15.129:0/940128010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618819d250 con 0x7f6188102790 2024-09-05T01:12:23.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f618cd99640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 0x7f61881982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6187fff640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f617400c920 tx=0x7f617400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174007c70 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6174007e10 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618819d540 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174018820 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.660+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61881a4e20 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.661+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61881021c0 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.662+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f61740189c0 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.663+0000 7f6185ffb640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 0x7f61780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.663+0000 7f618cd99640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 0x7f61780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.663+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f61740ced60 con 0x7f6188102790 2024-09-05T01:12:23.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.664+0000 7f618cd99640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 0x7f61780a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f617c0058c0 tx=0x7f617c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:23.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.670+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6174014030 con 0x7f6188102790 2024-09-05T01:12:23.865 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:23 smithi194 ceph-mon[34941]: Deploying daemon mds.l on smithi194 2024-09-05T01:12:23.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.872+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f618802d7c0 con 0x7f6188102790 2024-09-05T01:12:23.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:23 smithi129 ceph-mon[30979]: Deploying daemon mds.l on smithi194 2024-09-05T01:12:23.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.873+0000 7f6185ffb640 1 -- 172.21.15.129:0/940128010 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f617409c160 con 0x7f6188102790 2024-09-05T01:12:23.874 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:23.874 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"btime":"2024-09-05T01:05:32:738936+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-05T01:12:23.874 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 1 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 msgr2=0x7f61780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 0x7f61780a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f617c0058c0 tx=0x7f617c005d30 comp rx=0 tx=0).stop 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 msgr2=0x7f61881987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f617400c920 tx=0x7f617400cde0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 shutdown_connections 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f61780a2b80 0x7f61780a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6188106f10 0x7f618819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6188102790 0x7f61881987e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 --2- 172.21.15.129:0/940128010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6188101de0 0x7f61881982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.875+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 >> 172.21.15.129:0/940128010 conn(0x7f61880fd8c0 msgr2=0x7f61880feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.876+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 shutdown_connections 2024-09-05T01:12:23.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:23.876+0000 7f618effd640 1 -- 172.21.15.129:0/940128010 wait complete. 2024-09-05T01:12:24.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:23 smithi183 ceph-mon[34589]: Deploying daemon mds.l on smithi194 2024-09-05T01:12:24.653 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 systemd[1]: Starting Ceph mds.l for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:24.904 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:24 smithi194 ceph-mon[34941]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:24.904 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:24 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/940128010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:24.904 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 podman[64758]: 2024-09-05 01:12:24.555133397 +0000 UTC m=+0.015872300 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:24.904 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 podman[64758]: 2024-09-05 01:12:24.667772577 +0000 UTC m=+0.128511466 container create 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-05T01:12:24.904 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 podman[64758]: 2024-09-05 01:12:24.904006057 +0000 UTC m=+0.364744947 container init 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, ceph=True, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T01:12:24.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:24 smithi129 ceph-mon[30979]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:24.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:24 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/940128010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:24 smithi183 ceph-mon[34589]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:24 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/940128010' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:25.251 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 podman[64758]: 2024-09-05 01:12:24.906764525 +0000 UTC m=+0.367503414 container start 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e) 2024-09-05T01:12:25.251 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 ceph-mds[64777]: -- 172.21.15.194:0/1654429190 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f3e90d860 con 0x561f3e8eb000 2024-09-05T01:12:25.251 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l[64773]: starting mds.l at 2024-09-05T01:12:25.251 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:25 smithi194 bash[64758]: 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 2024-09-05T01:12:25.251 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:12:25 smithi194 systemd[1]: Started Ceph mds.l for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6832/1551944445,v1:172.21.15.194:6833/1551944445] up:boot 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: fsmap 1 up:standby 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:26.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:26.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: Deploying daemon mds.k on smithi183 2024-09-05T01:12:26.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:25 smithi183 ceph-mon[34589]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6832/1551944445,v1:172.21.15.194:6833/1551944445] up:boot 2024-09-05T01:12:26.249 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: fsmap 1 up:standby 2024-09-05T01:12:26.249 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:12:26.249 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.249 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: Deploying daemon mds.k on smithi183 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:25 smithi129 ceph-mon[30979]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6832/1551944445,v1:172.21.15.194:6833/1551944445] up:boot 2024-09-05T01:12:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: fsmap 1 up:standby 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: Deploying daemon mds.k on smithi183 2024-09-05T01:12:26.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:25 smithi194 ceph-mon[34941]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:26.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:26.883 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:27.331 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 systemd[1]: Starting Ceph mds.k for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/2344244173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8100e20 msgr2=0x7fdbb8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/2344244173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8100e20 0x7fdbb8101200 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802a480 comp rx=0 tx=0).stop 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/2344244173 shutdown_connections 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/2344244173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102ef0 0x7fdbb810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/2344244173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8102570 0x7fdbb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/2344244173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8100e20 0x7fdbb8101200 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/2344244173 >> 172.21.15.129:0/2344244173 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/2344244173 shutdown_connections 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.374+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/2344244173 wait complete. 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.375+0000 7fdbbd0c8640 1 Processor -- start 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.375+0000 7fdbbd0c8640 1 -- start start 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102570 0x7fdbb81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8110e30 con 0x7fdbb8102ef0 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8110cb0 con 0x7fdbb8100e20 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb8110fb0 con 0x7fdbb8102570 2024-09-05T01:12:27.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40740/0 (socket says 172.21.15.129:40740) 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:38312/0 (socket says 172.21.15.129:38312) 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 -- 172.21.15.129:0/235927458 learned_addr learned my addr 172.21.15.129:0/235927458 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb659c640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102570 0x7fdbb81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 -- 172.21.15.129:0/235927458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102570 msgr2=0x7fdbb81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102570 0x7fdbb81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 -- 172.21.15.129:0/235927458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 msgr2=0x7fdbb81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 -- 172.21.15.129:0/235927458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb819d360 con 0x7fdbb8102ef0 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb6d9d640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:12:27.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbb759e640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000cce0 tx=0x7fdbb000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb000f450 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb819d650 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbb000faa0 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0018b70 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.376+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a4f30 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.377+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbb8101200 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.378+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdbb0018d10 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.379+0000 7fdbaffff640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 0x7fdba40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.379+0000 7fdbb6d9d640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 0x7fdba40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.379+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fdbb00cef40 con 0x7fdbb8102ef0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.380+0000 7fdbb6d9d640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 0x7fdba40a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009230 tx=0x7fdba80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:27.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.383+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbb0014030 con 0x7fdbb8102ef0 2024-09-05T01:12:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.582+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbb8199710 con 0x7fdbb8102ef0 2024-09-05T01:12:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.582+0000 7fdbaffff640 1 -- 172.21.15.129:0/235927458 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7fdbb009c340 con 0x7fdbb8102ef0 2024-09-05T01:12:27.583 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:27.583 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"btime":"2024-09-05T01:12:24:934576+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-05T01:12:27.583 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 2 2024-09-05T01:12:27.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 msgr2=0x7fdba40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:27.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 0x7fdba40a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009230 tx=0x7fdba80058d0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 msgr2=0x7fdbb819cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:27.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000cce0 tx=0x7fdbb000a620 comp rx=0 tx=0).stop 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 shutdown_connections 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdba40a2b80 0x7fdba40a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb8102ef0 0x7fdbb819cc80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb8102570 0x7fdbb81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 --2- 172.21.15.129:0/235927458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb8100e20 0x7fdbb81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 >> 172.21.15.129:0/235927458 conn(0x7fdbb80fc7d0 msgr2=0x7fdbb806a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 shutdown_connections 2024-09-05T01:12:27.586 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:27.585+0000 7fdbbd0c8640 1 -- 172.21.15.129:0/235927458 wait complete. 2024-09-05T01:12:27.635 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 podman[63679]: 2024-09-05 01:12:27.233097249 +0000 UTC m=+0.016084404 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:27.635 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 podman[63679]: 2024-09-05 01:12:27.378369696 +0000 UTC m=+0.161356860 container create 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:27 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/235927458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 podman[63679]: 2024-09-05 01:12:27.65919781 +0000 UTC m=+0.442184985 container init 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e) 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 podman[63679]: 2024-09-05 01:12:27.662643121 +0000 UTC m=+0.445630294 container start 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 ceph-mds[63707]: -- 172.21.15.183:0/3952255574 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644af011860 con 0x5644aefef400 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k[63698]: starting mds.k at 2024-09-05T01:12:27.911 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 bash[63679]: 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 2024-09-05T01:12:27.912 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:12:27 smithi183 systemd[1]: Started Ceph mds.k for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:27.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:27 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/235927458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:28.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:27 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/235927458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:28.915 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:28.916 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:28.916 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: Deploying daemon mds.j on smithi129 2024-09-05T01:12:28.916 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6832/263217116,v1:172.21.15.183:6833/263217116] up:boot 2024-09-05T01:12:28.916 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: fsmap 2 up:standby 2024-09-05T01:12:28.916 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:28 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: Deploying daemon mds.j on smithi129 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6832/263217116,v1:172.21.15.183:6833/263217116] up:boot 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: fsmap 2 up:standby 2024-09-05T01:12:29.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:28 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: Deploying daemon mds.j on smithi129 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6832/263217116,v1:172.21.15.183:6833/263217116] up:boot 2024-09-05T01:12:29.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: fsmap 2 up:standby 2024-09-05T01:12:29.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:28 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T01:12:30.242 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 systemd[1]: Starting Ceph mds.j for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:30.380 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:30.535 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 podman[69971]: 2024-09-05 01:12:30.180848226 +0000 UTC m=+0.016147725 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:30.535 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 podman[69971]: 2024-09-05 01:12:30.288361903 +0000 UTC m=+0.123661392 container create efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T01:12:30.535 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 podman[69971]: 2024-09-05 01:12:30.508097404 +0000 UTC m=+0.343396894 container init efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-05T01:12:30.535 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 podman[69971]: 2024-09-05 01:12:30.511172358 +0000 UTC m=+0.346471853 container start efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T01:12:30.694 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:30.794 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mds[70008]: -- 172.21.15.129:0/3117873242 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55898dd1f860 con 0x55898dcfc800 2024-09-05T01:12:30.794 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j[69999]: starting mds.j at 2024-09-05T01:12:30.794 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 bash[69971]: efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 2024-09-05T01:12:30.794 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:12:30 smithi129 systemd[1]: Started Ceph mds.j for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:31.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:31.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.088 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:31.088 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:31.088 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:30 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:30 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 -- 172.21.15.129:0/617790965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418102790 msgr2=0x7fd4181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 --2- 172.21.15.129:0/617790965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418102790 0x7fd4181067e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40002a410 comp rx=0 tx=0).stop 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 -- 172.21.15.129:0/617790965 shutdown_connections 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 --2- 172.21.15.129:0/617790965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418106f10 0x7fd41810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 --2- 172.21.15.129:0/617790965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418102790 0x7fd4181067e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 --2- 172.21.15.129:0/617790965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418101de0 0x7fd4181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 -- 172.21.15.129:0/617790965 >> 172.21.15.129:0/617790965 conn(0x7fd4180fd8c0 msgr2=0x7fd4180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 -- 172.21.15.129:0/617790965 shutdown_connections 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.167+0000 7fd41cdca640 1 -- 172.21.15.129:0/617790965 wait complete. 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 Processor -- start 2024-09-05T01:12:31.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 -- start start 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418101de0 0x7fd418198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418102790 0x7fd418198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd418111e40 con 0x7fd418106f10 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd418111cc0 con 0x7fd418101de0 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd41cdca640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd418111fc0 con 0x7fd418102790 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.168+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40782/0 (socket says 172.21.15.129:40782) 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 -- 172.21.15.129:0/3964999897 learned_addr learned my addr 172.21.15.129:0/3964999897 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd41659c640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418101de0 0x7fd418198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:31.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd40ffff640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418102790 0x7fd418198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 -- 172.21.15.129:0/3964999897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418102790 msgr2=0x7fd418198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418102790 0x7fd418198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 -- 172.21.15.129:0/3964999897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418101de0 msgr2=0x7fd418198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418101de0 0x7fd418198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 -- 172.21.15.129:0/3964999897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819d3d0 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd416d9d640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd41000eaa0 tx=0x7fd41000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410004d20 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819d6c0 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd410005020 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4100116b0 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.169+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181a4fa0 con 0x7fd418106f10 2024-09-05T01:12:31.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.170+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4181021c0 con 0x7fd418106f10 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.171+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd4100051d0 con 0x7fd418106f10 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.172+0000 7fd40f7fe640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 0x7fd4040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.172+0000 7fd41659c640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 0x7fd4040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.172+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fd4100cede0 con 0x7fd418106f10 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.173+0000 7fd41659c640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 0x7fd4040a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd408009990 tx=0x7fd408009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:31.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.177+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd410016030 con 0x7fd418106f10 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:30 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:31.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.389+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd41810a150 con 0x7fd418106f10 2024-09-05T01:12:31.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.390+0000 7fd40f7fe640 1 -- 172.21.15.129:0/3964999897 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7fd41009c1e0 con 0x7fd418106f10 2024-09-05T01:12:31.391 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:31.391 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":4,"btime":"2024-09-05T01:12:30:819849+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T01:12:31.391 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 4 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 msgr2=0x7fd4040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 0x7fd4040a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd408009990 tx=0x7fd408009110 comp rx=0 tx=0).stop 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 msgr2=0x7fd41819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd41000eaa0 tx=0x7fd41000eea0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 shutdown_connections 2024-09-05T01:12:31.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd4040a2b80 0x7fd4040a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd418106f10 0x7fd41819ccf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd418102790 0x7fd418198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 --2- 172.21.15.129:0/3964999897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd418101de0 0x7fd418198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 >> 172.21.15.129:0/3964999897 conn(0x7fd4180fd8c0 msgr2=0x7fd4180fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 shutdown_connections 2024-09-05T01:12:31.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:31.392+0000 7fd41cdca640 1 -- 172.21.15.129:0/3964999897 wait complete. 2024-09-05T01:12:32.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:31 smithi183 ceph-mon[34589]: Deploying daemon mds.d on smithi129 2024-09-05T01:12:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.129:6834/2733798787,v1:172.21.15.129:6835/2733798787] up:boot 2024-09-05T01:12:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:31 smithi183 ceph-mon[34589]: fsmap 3 up:standby 2024-09-05T01:12:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:12:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:31 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3964999897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:31 smithi129 ceph-mon[30979]: Deploying daemon mds.d on smithi129 2024-09-05T01:12:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.129:6834/2733798787,v1:172.21.15.129:6835/2733798787] up:boot 2024-09-05T01:12:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:31 smithi129 ceph-mon[30979]: fsmap 3 up:standby 2024-09-05T01:12:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:12:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:31 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3964999897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:31 smithi194 ceph-mon[34941]: Deploying daemon mds.d on smithi129 2024-09-05T01:12:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.129:6834/2733798787,v1:172.21.15.129:6835/2733798787] up:boot 2024-09-05T01:12:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:31 smithi194 ceph-mon[34941]: fsmap 3 up:standby 2024-09-05T01:12:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T01:12:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:31 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3964999897' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:32 smithi183 ceph-mon[34589]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:32 smithi129 ceph-mon[30979]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:32 smithi194 ceph-mon[34941]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:33.976 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:33 smithi129 systemd[1]: Starting Ceph mds.d for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:34.242 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:33 smithi129 podman[70387]: 2024-09-05 01:12:33.878690776 +0000 UTC m=+0.016038214 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:34.243 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 podman[70387]: 2024-09-05 01:12:34.11521709 +0000 UTC m=+0.252564524 container create 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T01:12:34.323 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:34.585 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 podman[70387]: 2024-09-05 01:12:34.377229938 +0000 UTC m=+0.514577390 container init 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 podman[70387]: 2024-09-05 01:12:34.380799265 +0000 UTC m=+0.518146697 container start 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mds[70424]: -- 172.21.15.129:0/3229908906 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e082949860 con 0x55e082926800 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d[70402]: starting mds.d at 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 bash[70387]: 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 2024-09-05T01:12:34.708 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:12:34 smithi129 systemd[1]: Started Ceph mds.d for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:34.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:34.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:34.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:34.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:34.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:34.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:34.974 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:34 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:35.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.052+0000 7fe3c53c3640 1 -- 172.21.15.129:0/441363940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c01027a0 msgr2=0x7fe3c01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:35.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.052+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/441363940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c01027a0 0x7fe3c01067f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009900 tx=0x7fe3b402a410 comp rx=0 tx=0).stop 2024-09-05T01:12:35.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 -- 172.21.15.129:0/441363940 shutdown_connections 2024-09-05T01:12:35.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/441363940 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0106f20 0x7fe3c010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/441363940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c01027a0 0x7fe3c01067f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/441363940 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c0101df0 0x7fe3c01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 -- 172.21.15.129:0/441363940 >> 172.21.15.129:0/441363940 conn(0x7fe3c00fd8b0 msgr2=0x7fe3c00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 -- 172.21.15.129:0/441363940 shutdown_connections 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 -- 172.21.15.129:0/441363940 wait complete. 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 Processor -- start 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.053+0000 7fe3c53c3640 1 -- start start 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0101df0 0x7fe3c01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 0x7fe3c0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0111e30 con 0x7fe3c0106f20 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0111cb0 con 0x7fe3c01027a0 2024-09-05T01:12:35.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0111fb0 con 0x7fe3c0101df0 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60000/0 (socket says 172.21.15.129:60000) 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 -- 172.21.15.129:0/3243762892 learned_addr learned my addr 172.21.15.129:0/3243762892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3be7fc640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 0x7fe3c0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3beffd640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0101df0 0x7fe3c01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 -- 172.21.15.129:0/3243762892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0101df0 msgr2=0x7fe3c01983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0101df0 0x7fe3c01983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 -- 172.21.15.129:0/3243762892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 msgr2=0x7fe3c0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 0x7fe3c0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 -- 172.21.15.129:0/3243762892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c019d3a0 con 0x7fe3c0106f20 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3be7fc640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 0x7fe3c0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:12:35.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3bf7fe640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800d750 tx=0x7fe3b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8017070 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c019d690 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b80054d0 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8004c60 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.054+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01a4f70 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.055+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c01021d0 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.057+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe3b8004e00 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.057+0000 7fe3a7fff640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 0x7fe3a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.057+0000 7fe3beffd640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 0x7fe3a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.057+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe3b80ce840 con 0x7fe3c0106f20 2024-09-05T01:12:35.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.058+0000 7fe3beffd640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 0x7fe3a80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0005fd0 tx=0x7fe3b0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:35.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.061+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3b809b5f0 con 0x7fe3c0106f20 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:35.127 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:34 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:35.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:34 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:35.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.262+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c010a160 con 0x7fe3c0106f20 2024-09-05T01:12:35.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.263+0000 7fe3a7fff640 1 -- 172.21.15.129:0/3243762892 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7fe3b809f8f0 con 0x7fe3c0106f20 2024-09-05T01:12:35.265 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:35.265 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":5,"btime":"2024-09-05T01:12:34:831929+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T01:12:35.265 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 5 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 msgr2=0x7fe3a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 0x7fe3a80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0005fd0 tx=0x7fe3b0009500 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 msgr2=0x7fe3c019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800d750 tx=0x7fe3b800dc10 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 shutdown_connections 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3a80a2b80 0x7fe3a80a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3c0106f20 0x7fe3c019ccc0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe3c01027a0 0x7fe3c0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 --2- 172.21.15.129:0/3243762892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe3c0101df0 0x7fe3c01983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 >> 172.21.15.129:0/3243762892 conn(0x7fe3c00fd8b0 msgr2=0x7fe3c00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:35.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 shutdown_connections 2024-09-05T01:12:35.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:35.265+0000 7fe3c53c3640 1 -- 172.21.15.129:0/3243762892 wait complete. 2024-09-05T01:12:35.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:35 smithi129 ceph-mon[30979]: Deploying daemon mds.e on smithi183 2024-09-05T01:12:35.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:35 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.129:6836/1098380818,v1:172.21.15.129:6837/1098380818] up:boot 2024-09-05T01:12:35.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:35 smithi129 ceph-mon[30979]: fsmap 4 up:standby 2024-09-05T01:12:35.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:35 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:12:35.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:35 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3243762892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:36.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:35 smithi183 ceph-mon[34589]: Deploying daemon mds.e on smithi183 2024-09-05T01:12:36.123 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:35 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.129:6836/1098380818,v1:172.21.15.129:6837/1098380818] up:boot 2024-09-05T01:12:36.124 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:35 smithi183 ceph-mon[34589]: fsmap 4 up:standby 2024-09-05T01:12:36.124 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:35 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:12:36.124 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:35 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3243762892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:36.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:35 smithi194 ceph-mon[34941]: Deploying daemon mds.e on smithi183 2024-09-05T01:12:36.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:35 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.129:6836/1098380818,v1:172.21.15.129:6837/1098380818] up:boot 2024-09-05T01:12:36.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:35 smithi194 ceph-mon[34941]: fsmap 4 up:standby 2024-09-05T01:12:36.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:35 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T01:12:36.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:35 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3243762892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:36.828 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:36 smithi183 systemd[1]: Starting Ceph mds.e for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:37.120 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:36 smithi183 ceph-mon[34589]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:37.120 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:36 smithi183 podman[63993]: 2024-09-05 01:12:36.730376039 +0000 UTC m=+0.015763815 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:37.120 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:36 smithi183 podman[63993]: 2024-09-05 01:12:36.84225879 +0000 UTC m=+0.127646556 container create 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-05T01:12:37.120 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 podman[63993]: 2024-09-05 01:12:37.120199536 +0000 UTC m=+0.405587301 container init 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-05T01:12:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:36 smithi129 ceph-mon[30979]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:36 smithi194 ceph-mon[34941]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:37.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 podman[63993]: 2024-09-05 01:12:37.123064625 +0000 UTC m=+0.408452391 container start 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-05T01:12:37.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 ceph-mds[64012]: -- 172.21.15.183:0/800704083 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561079d5860 con 0x5561079b3400 2024-09-05T01:12:37.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: starting mds.e at 2024-09-05T01:12:37.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 bash[63993]: 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d 2024-09-05T01:12:37.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:12:37 smithi183 systemd[1]: Started Ceph mds.e for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:38.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:38.369 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: Deploying daemon mds.f on smithi194 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:38.491 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6834/3903364483,v1:172.21.15.183:6835/3903364483] up:boot 2024-09-05T01:12:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: fsmap 5 up:standby 2024-09-05T01:12:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:38 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: Deploying daemon mds.f on smithi194 2024-09-05T01:12:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:38.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6834/3903364483,v1:172.21.15.183:6835/3903364483] up:boot 2024-09-05T01:12:38.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: fsmap 5 up:standby 2024-09-05T01:12:38.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:38 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:12:38.736 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.736 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.736 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: Deploying daemon mds.f on smithi194 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6834/3903364483,v1:172.21.15.183:6835/3903364483] up:boot 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: fsmap 5 up:standby 2024-09-05T01:12:38.737 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:38 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:12:38.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 -- 172.21.15.129:0/3330836221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4101de0 msgr2=0x7f0fe41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:38.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 --2- 172.21.15.129:0/3330836221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4101de0 0x7f0fe41021c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009900 tx=0x7f0fd402a410 comp rx=0 tx=0).stop 2024-09-05T01:12:38.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 -- 172.21.15.129:0/3330836221 shutdown_connections 2024-09-05T01:12:38.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 --2- 172.21.15.129:0/3330836221 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4106f10 0x7f0fe410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:38.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 --2- 172.21.15.129:0/3330836221 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 --2- 172.21.15.129:0/3330836221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4101de0 0x7f0fe41021c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 -- 172.21.15.129:0/3330836221 >> 172.21.15.129:0/3330836221 conn(0x7f0fe40fd8c0 msgr2=0x7f0fe40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 -- 172.21.15.129:0/3330836221 shutdown_connections 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.870+0000 7f0fe975a640 1 -- 172.21.15.129:0/3330836221 wait complete. 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.871+0000 7f0fe975a640 1 Processor -- start 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.871+0000 7f0fe975a640 1 -- start start 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4101de0 0x7f0fe4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4111e20 con 0x7f0fe4106f10 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4111ca0 con 0x7f0fe4101de0 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4111fa0 con 0x7f0fe4102790 2024-09-05T01:12:38.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60024/0 (socket says 172.21.15.129:60024) 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 -- 172.21.15.129:0/1842340710 learned_addr learned my addr 172.21.15.129:0/1842340710 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4101de0 0x7f0fe4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe27fc640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 -- 172.21.15.129:0/1842340710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 msgr2=0x7f0fe41988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 -- 172.21.15.129:0/1842340710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4101de0 msgr2=0x7f0fe4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4101de0 0x7f0fe4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 -- 172.21.15.129:0/1842340710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe419d330 con 0x7f0fe4106f10 2024-09-05T01:12:38.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe27fc640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe37fe640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00eaa0 tx=0x7f0fdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00cdb0 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe419d620 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fdc005020 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.872+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe41a4f00 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.873+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc01f060 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.873+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe41021c0 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.874+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0fdc0051d0 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.875+0000 7f0fcbfff640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 0x7f0fcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.875+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 0x7f0fcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.875+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0fdc0ced00 con 0x7f0fe4106f10 2024-09-05T01:12:38.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.876+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 0x7f0fcc0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4031440 tx=0x7f0fd4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:38.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:38.879+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fdc016030 con 0x7f0fe4106f10 2024-09-05T01:12:39.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.082+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0fe402d7c0 con 0x7f0fe4106f10 2024-09-05T01:12:39.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.082+0000 7f0fcbfff640 1 -- 172.21.15.129:0/1842340710 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7f0fdc09c100 con 0x7f0fe4106f10 2024-09-05T01:12:39.084 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:39.084 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":6,"btime":"2024-09-05T01:12:37:845310+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T01:12:39.085 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 6 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 msgr2=0x7f0fcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 0x7f0fcc0a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4031440 tx=0x7f0fd4005b50 comp rx=0 tx=0).stop 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 msgr2=0x7f0fe419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00eaa0 tx=0x7f0fdc00ef60 comp rx=0 tx=0).stop 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 shutdown_connections 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fcc0a2b80 0x7f0fcc0a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4106f10 0x7f0fe419cc50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4102790 0x7f0fe41988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:39.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 --2- 172.21.15.129:0/1842340710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4101de0 0x7f0fe4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:39.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.084+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 >> 172.21.15.129:0/1842340710 conn(0x7f0fe40fd8c0 msgr2=0x7f0fe40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:39.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.085+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 shutdown_connections 2024-09-05T01:12:39.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:39.085+0000 7f0fe975a640 1 -- 172.21.15.129:0/1842340710 wait complete. 2024-09-05T01:12:39.297 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 systemd[1]: Starting Ceph mds.f for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:39.463 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1842340710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:39.559 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:39 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1842340710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:39.560 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 podman[65061]: 2024-09-05 01:12:39.321426656 +0000 UTC m=+0.015819007 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:39.560 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 podman[65061]: 2024-09-05 01:12:39.438779875 +0000 UTC m=+0.133172214 container create f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T01:12:39.660 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:39 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1842340710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:40.000 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 podman[65061]: 2024-09-05 01:12:39.691474488 +0000 UTC m=+0.385866821 container init f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T01:12:40.000 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 podman[65061]: 2024-09-05 01:12:39.694557888 +0000 UTC m=+0.388950231 container start f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD) 2024-09-05T01:12:40.000 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 ceph-mds[65080]: -- 172.21.15.194:0/3488540163 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a2437d860 con 0x559a2435b400 2024-09-05T01:12:40.000 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: starting mds.f at 2024-09-05T01:12:40.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 bash[65061]: f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b 2024-09-05T01:12:40.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:12:39 smithi194 systemd[1]: Started Ceph mds.f for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:41.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: Deploying daemon mds.c on smithi194 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6834/1641200590,v1:172.21.15.194:6835/1641200590] up:boot 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: fsmap 6 up:standby 2024-09-05T01:12:41.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:40 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: Deploying daemon mds.c on smithi194 2024-09-05T01:12:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6834/1641200590,v1:172.21.15.194:6835/1641200590] up:boot 2024-09-05T01:12:41.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: fsmap 6 up:standby 2024-09-05T01:12:41.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:40 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:41.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: Deploying daemon mds.c on smithi194 2024-09-05T01:12:41.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6834/1641200590,v1:172.21.15.194:6835/1641200590] up:boot 2024-09-05T01:12:41.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: fsmap 6 up:standby 2024-09-05T01:12:41.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:40 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:12:41.927 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:41.992 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:41 smithi194 systemd[1]: Starting Ceph mds.c for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:42.190 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:42.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:41 smithi194 podman[65358]: 2024-09-05 01:12:41.894195805 +0000 UTC m=+0.015706366 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:42.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 podman[65358]: 2024-09-05 01:12:42.018928484 +0000 UTC m=+0.140439042 container create ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 -- 172.21.15.129:0/769413543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268101de0 msgr2=0x7f42681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 --2- 172.21.15.129:0/769413543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268101de0 0x7f42681021c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4258009900 tx=0x7f425802a410 comp rx=0 tx=0).stop 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 -- 172.21.15.129:0/769413543 shutdown_connections 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 --2- 172.21.15.129:0/769413543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268106f10 0x7f426810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 --2- 172.21.15.129:0/769413543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268102790 0x7f42681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 --2- 172.21.15.129:0/769413543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268101de0 0x7f42681021c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 -- 172.21.15.129:0/769413543 >> 172.21.15.129:0/769413543 conn(0x7f42680fd8c0 msgr2=0x7f42680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:42.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 -- 172.21.15.129:0/769413543 shutdown_connections 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.675+0000 7f426d28d640 1 -- 172.21.15.129:0/769413543 wait complete. 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 Processor -- start 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 -- start start 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268102790 0x7f4268198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268106f10 0x7f426819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4268111e40 con 0x7f4268106f10 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4268111cc0 con 0x7f4268101de0 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f426d28d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4268111fc0 con 0x7f4268102790 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f4266ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f4266ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56990/0 (socket says 172.21.15.129:56990) 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f4266ffd640 1 -- 172.21.15.129:0/3802119365 learned_addr learned my addr 172.21.15.129:0/3802119365 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:42.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.676+0000 7f42677fe640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268106f10 0x7f426819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f425ffff640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268102790 0x7f4268198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 -- 172.21.15.129:0/3802119365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268102790 msgr2=0x7f4268198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268102790 0x7f4268198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 -- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268106f10 msgr2=0x7f426819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268106f10 0x7f426819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 -- 172.21.15.129:0/3802119365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426819d370 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4266ffd640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4258005940 tx=0x7f42580042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425803f030 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426819d600 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4258003cb0 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258035070 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.677+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a4f40 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.678+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42681032c0 con 0x7f4268101de0 2024-09-05T01:12:42.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.679+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4258042030 con 0x7f4268101de0 2024-09-05T01:12:42.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.680+0000 7f4264ff9640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 0x7f42540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:42.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.680+0000 7f425ffff640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 0x7f42540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:42.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.680+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f42580ef660 con 0x7f4268101de0 2024-09-05T01:12:42.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.680+0000 7f425ffff640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 0x7f42540a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42680ffe00 tx=0x7f4250005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:42.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.683+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42580bc9c0 con 0x7f4268101de0 2024-09-05T01:12:42.750 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 podman[65358]: 2024-09-05 01:12:42.296949555 +0000 UTC m=+0.418460125 container init ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-05T01:12:42.750 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 podman[65358]: 2024-09-05 01:12:42.299837123 +0000 UTC m=+0.421347690 container start ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-05T01:12:42.750 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mds[65377]: -- 172.21.15.194:0/582605997 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d6c837860 con 0x555d6c814800 2024-09-05T01:12:42.750 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: starting mds.c at 2024-09-05T01:12:42.751 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 bash[65358]: ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb 2024-09-05T01:12:42.751 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:12:42 smithi194 systemd[1]: Started Ceph mds.c for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:42.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.894+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f426810a0f0 con 0x7f4268101de0 2024-09-05T01:12:42.897 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.895+0000 7f4264ff9640 1 -- 172.21.15.129:0/3802119365 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6400 (secure 0 0 0) 0x7f42580bc2c0 con 0x7f4268101de0 2024-09-05T01:12:42.897 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 8 2024-09-05T01:12:42.897 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:42.898 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":8,"btime":"2024-09-05T01:12:42:868577+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-05T01:12:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 msgr2=0x7f42540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 0x7f42540a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f42680ffe00 tx=0x7f4250005f90 comp rx=0 tx=0).stop 2024-09-05T01:12:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 msgr2=0x7f42681983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4258005940 tx=0x7f42580042f0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.899 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 shutdown_connections 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42540a2b80 0x7f42540a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4268106f10 0x7f426819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4268102790 0x7f4268198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 --2- 172.21.15.129:0/3802119365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4268101de0 0x7f42681983c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.897+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 >> 172.21.15.129:0/3802119365 conn(0x7f42680fd8c0 msgr2=0x7f42680fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.898+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 shutdown_connections 2024-09-05T01:12:42.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:42.898+0000 7f426d28d640 1 -- 172.21.15.129:0/3802119365 wait complete. 2024-09-05T01:12:43.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:43.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.140 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:43.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:43.141 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:42 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:42 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:42 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:43 smithi183 ceph-mon[34589]: Deploying daemon mds.h on smithi183 2024-09-05T01:12:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:43 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6836/979076613,v1:172.21.15.194:6837/979076613] up:boot 2024-09-05T01:12:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:43 smithi183 ceph-mon[34589]: fsmap 7 up:standby 2024-09-05T01:12:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:43 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:12:44.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:43 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3802119365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:43 smithi129 ceph-mon[30979]: Deploying daemon mds.h on smithi183 2024-09-05T01:12:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:43 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6836/979076613,v1:172.21.15.194:6837/979076613] up:boot 2024-09-05T01:12:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:43 smithi129 ceph-mon[30979]: fsmap 7 up:standby 2024-09-05T01:12:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:43 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:12:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:43 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3802119365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:43 smithi194 ceph-mon[34941]: Deploying daemon mds.h on smithi183 2024-09-05T01:12:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:43 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6836/979076613,v1:172.21.15.194:6837/979076613] up:boot 2024-09-05T01:12:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:43 smithi194 ceph-mon[34941]: fsmap 7 up:standby 2024-09-05T01:12:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:43 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:12:44.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:43 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3802119365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:44.661 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:44 smithi183 systemd[1]: Starting Ceph mds.h for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:44.984 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:44 smithi183 ceph-mon[34589]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:44.984 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:44 smithi183 podman[64296]: 2024-09-05 01:12:44.587843777 +0000 UTC m=+0.016056477 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:44.985 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:44 smithi183 podman[64296]: 2024-09-05 01:12:44.714990848 +0000 UTC m=+0.143203598 container create 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-05T01:12:44.985 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:44 smithi183 podman[64296]: 2024-09-05 01:12:44.984476004 +0000 UTC m=+0.412688685 container init 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-91c8a7e, ceph=True, io.buildah.version=1.37.2) 2024-09-05T01:12:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:44 smithi129 ceph-mon[30979]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:44 smithi194 ceph-mon[34941]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:45.411 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:44 smithi183 podman[64296]: 2024-09-05 01:12:44.987640256 +0000 UTC m=+0.415852948 container start 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T01:12:45.411 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:45 smithi183 ceph-mds[64315]: -- 172.21.15.183:0/3687031427 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cd7dff860 con 0x564cd7ddd000 2024-09-05T01:12:45.411 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: starting mds.h at 2024-09-05T01:12:45.411 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:45 smithi183 bash[64296]: 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a 2024-09-05T01:12:45.411 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:12:45 smithi183 systemd[1]: Started Ceph mds.h for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:45.538 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:45.799 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: Deploying daemon mds.i on smithi194 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:boot 2024-09-05T01:12:46.212 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: fsmap 8 up:standby 2024-09-05T01:12:46.213 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:46 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.291+0000 7ff023b03640 1 -- 172.21.15.129:0/824871736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1036e0 msgr2=0x7ff01c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.291+0000 7ff023b03640 1 --2- 172.21.15.129:0/824871736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1036e0 0x7ff01c107b90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff010009930 tx=0x7ff01002a420 comp rx=0 tx=0).stop 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 -- 172.21.15.129:0/824871736 shutdown_connections 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- 172.21.15.129:0/824871736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1082a0 0x7ff01c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- 172.21.15.129:0/824871736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1036e0 0x7ff01c107b90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- 172.21.15.129:0/824871736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 0x7ff01c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 -- 172.21.15.129:0/824871736 >> 172.21.15.129:0/824871736 conn(0x7ff01c0fc7d0 msgr2=0x7ff01c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 -- 172.21.15.129:0/824871736 shutdown_connections 2024-09-05T01:12:46.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 -- 172.21.15.129:0/824871736 wait complete. 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 Processor -- start 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 -- start start 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 0x7ff01c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1036e0 0x7ff01c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.292+0000 7ff023b03640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff023b03640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c111f10 con 0x7ff01c1082a0 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff023b03640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c111d90 con 0x7ff01c1036e0 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff023b03640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c112090 con 0x7ff01c102d30 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff02189f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 0x7ff01c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58436/0 (socket says 172.21.15.129:58436) 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 -- 172.21.15.129:0/2874719848 learned_addr learned my addr 172.21.15.129:0/2874719848 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:46.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff02109e640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1036e0 0x7ff01c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 -- 172.21.15.129:0/2874719848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 msgr2=0x7ff01c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 0x7ff01c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 -- 172.21.15.129:0/2874719848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1036e0 msgr2=0x7ff01c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1036e0 0x7ff01c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 -- 172.21.15.129:0/2874719848 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c1a6910 con 0x7ff01c1082a0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff0220a0640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff01400e970 tx=0x7ff01400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01400ccc0 con 0x7ff01c1082a0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff01400ce60 con 0x7ff01c1082a0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1a6c00 con 0x7ff01c1082a0 2024-09-05T01:12:46.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014010430 con 0x7ff01c1082a0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.293+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1a70c0 con 0x7ff01c1082a0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.294+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c103110 con 0x7ff01c1082a0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.295+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff014004ff0 con 0x7ff01c1082a0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.296+0000 7ff01affd640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.296+0000 7ff02189f640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.296+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff0140ceff0 con 0x7ff01c1082a0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.297+0000 7ff02189f640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 0x7ff0040a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007ea0 tx=0x7ff00c0075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:46.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.300+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff01409c3f0 con 0x7ff01c1082a0 2024-09-05T01:12:46.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: Deploying daemon mds.i on smithi194 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:boot 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: fsmap 8 up:standby 2024-09-05T01:12:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:46 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: Deploying daemon mds.i on smithi194 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:boot 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: fsmap 8 up:standby 2024-09-05T01:12:46.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:46 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:12:46.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.501+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff01c10a2b0 con 0x7ff01c1082a0 2024-09-05T01:12:46.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.502+0000 7ff01affd640 1 -- 172.21.15.129:0/2874719848 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7225 (secure 0 0 0) 0x7ff01409bcf0 con 0x7ff01c1082a0 2024-09-05T01:12:46.503 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:46.503 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":9,"btime":"2024-09-05T01:12:45:883822+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T01:12:46.503 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 9 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 msgr2=0x7ff0040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 0x7ff0040a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007ea0 tx=0x7ff00c0075b0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 msgr2=0x7ff01c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff01400e970 tx=0x7ff01400ed70 comp rx=0 tx=0).stop 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 shutdown_connections 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff01c1082a0 0x7ff01c101410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff01c1036e0 0x7ff01c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 --2- 172.21.15.129:0/2874719848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff01c102d30 0x7ff01c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:46.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 >> 172.21.15.129:0/2874719848 conn(0x7ff01c0fc7d0 msgr2=0x7ff01c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:46.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 shutdown_connections 2024-09-05T01:12:46.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:46.504+0000 7ff023b03640 1 -- 172.21.15.129:0/2874719848 wait complete. 2024-09-05T01:12:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2874719848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:47 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2874719848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:47.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:47 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2874719848' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:48.000 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:12:47 smithi194 systemd[1]: Started Ceph mds.i for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:49.113 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:49.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:49.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: Deploying daemon mds.g on smithi129 2024-09-05T01:12:49.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:boot 2024-09-05T01:12:49.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: fsmap 9 up:standby 2024-09-05T01:12:49.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:48 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: Deploying daemon mds.g on smithi129 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:boot 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: fsmap 9 up:standby 2024-09-05T01:12:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:48 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:12:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: Deploying daemon mds.g on smithi129 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:boot 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: fsmap 9 up:standby 2024-09-05T01:12:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:48 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:12:49.366 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 -- 172.21.15.129:0/3819973235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c106f10 msgr2=0x7f425c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 --2- 172.21.15.129:0/3819973235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c106f10 0x7f425c10ea90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4254009900 tx=0x7f425402a410 comp rx=0 tx=0).stop 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 -- 172.21.15.129:0/3819973235 shutdown_connections 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 --2- 172.21.15.129:0/3819973235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c106f10 0x7f425c10ea90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 --2- 172.21.15.129:0/3819973235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c102790 0x7f425c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 --2- 172.21.15.129:0/3819973235 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f425c101de0 0x7f425c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 -- 172.21.15.129:0/3819973235 >> 172.21.15.129:0/3819973235 conn(0x7f425c0fd8c0 msgr2=0x7f425c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 -- 172.21.15.129:0/3819973235 shutdown_connections 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.006+0000 7f4260f5e640 1 -- 172.21.15.129:0/3819973235 wait complete. 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 Processor -- start 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 -- start start 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 0x7f425c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f425c106f10 0x7f425c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f425c111e20 con 0x7f425c102790 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f425c111ca0 con 0x7f425c101de0 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4260f5e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f425c111fa0 con 0x7f425c106f10 2024-09-05T01:12:50.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58468/0 (socket says 172.21.15.129:58468) 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 -- 172.21.15.129:0/1544773304 learned_addr learned my addr 172.21.15.129:0/1544773304 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 -- 172.21.15.129:0/1544773304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f425c106f10 msgr2=0x7f425c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f425c106f10 0x7f425c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 -- 172.21.15.129:0/1544773304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 msgr2=0x7f425c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f425a59c640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 0x7f425c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 0x7f425c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f4259d9b640 1 -- 172.21.15.129:0/1544773304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c19d2f0 con 0x7f425c102790 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.007+0000 7f425a59c640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 0x7f425c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f4259d9b640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9f0 tx=0x7f424400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424400cd80 con 0x7f425c102790 2024-09-05T01:12:50.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c19d5e0 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42440108e0 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4244018920 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.008+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a4ec0 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.009+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c1021c0 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.010+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4244010a90 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.011+0000 7f42537fe640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 0x7f42480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.011+0000 7f425a59c640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 0x7f42480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.011+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f42440cf480 con 0x7f425c102790 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.011+0000 7f425a59c640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 0x7f42480a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f424c005fd0 tx=0x7f424c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:50.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.014+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f424409c880 con 0x7f425c102790 2024-09-05T01:12:50.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.229+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f425c109b60 con 0x7f425c102790 2024-09-05T01:12:50.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.230+0000 7f42537fe640 1 -- 172.21.15.129:0/1544773304 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7f424409c180 con 0x7f425c102790 2024-09-05T01:12:50.231 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:50.231 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":10,"btime":"2024-09-05T01:12:48:150945+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T01:12:50.231 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 10 2024-09-05T01:12:50.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 msgr2=0x7f42480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 0x7f42480a5040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f424c005fd0 tx=0x7f424c0094d0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 msgr2=0x7f425c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f424400e9f0 tx=0x7f424400eeb0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 shutdown_connections 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f42480a2b80 0x7f42480a5040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f425c106f10 0x7f425c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f425c102790 0x7f425c198880 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 --2- 172.21.15.129:0/1544773304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f425c101de0 0x7f425c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 >> 172.21.15.129:0/1544773304 conn(0x7f425c0fd8c0 msgr2=0x7f425c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 shutdown_connections 2024-09-05T01:12:50.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:50.232+0000 7f4260f5e640 1 -- 172.21.15.129:0/1544773304 wait complete. 2024-09-05T01:12:50.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:50 smithi129 ceph-mon[30979]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:50.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:50 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1544773304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:50 smithi183 ceph-mon[34589]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:50 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1544773304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:50 smithi194 ceph-mon[34941]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:50 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1544773304' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.129:6838/2120045366,v1:172.21.15.129:6839/2120045366] up:boot 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: fsmap 10 up:standby 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:51 smithi183 ceph-mon[34589]: Deploying daemon mds.a on smithi129 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.129:6838/2120045366,v1:172.21.15.129:6839/2120045366] up:boot 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: fsmap 10 up:standby 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:52.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:51 smithi129 ceph-mon[30979]: Deploying daemon mds.a on smithi129 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.129:6838/2120045366,v1:172.21.15.129:6839/2120045366] up:boot 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: fsmap 10 up:standby 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:52.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:51 smithi194 ceph-mon[34941]: Deploying daemon mds.a on smithi129 2024-09-05T01:12:52.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:52 smithi129 ceph-mon[30979]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:52.992 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:52 smithi129 systemd[1]: Starting Ceph mds.a for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:52 smithi183 ceph-mon[34589]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:53.198 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:52 smithi194 ceph-mon[34941]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:53.325 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 podman[71577]: 2024-09-05 01:12:52.95738067 +0000 UTC m=+0.015652852 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:53.325 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 podman[71577]: 2024-09-05 01:12:53.09721951 +0000 UTC m=+0.155491694 container create f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-05T01:12:53.325 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 podman[71577]: 2024-09-05 01:12:53.325426452 +0000 UTC m=+0.383698637 container init f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T01:12:53.494 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:53.602 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 podman[71577]: 2024-09-05 01:12:53.328682297 +0000 UTC m=+0.386954488 container start f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:12:53.602 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 ceph-mds[71614]: -- 172.21.15.129:0/233797345 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594e20b7860 con 0x5594e2095400 2024-09-05T01:12:53.602 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a[71608]: starting mds.a at 2024-09-05T01:12:53.602 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 bash[71577]: f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 2024-09-05T01:12:53.602 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:12:53 smithi129 systemd[1]: Started Ceph mds.a for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.943+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3885078165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 msgr2=0x7f54c8100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.943+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3885078165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c8100090 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f54bc009900 tx=0x7f54bc02a400 comp rx=0 tx=0).stop 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3885078165 shutdown_connections 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3885078165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c8110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3885078165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c8102fc0 0x7f54c81005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3885078165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c8100090 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3885078165 >> 172.21.15.129:0/3885078165 conn(0x7f54c8077210 msgr2=0x7f54c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3885078165 shutdown_connections 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3885078165 wait complete. 2024-09-05T01:12:53.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 Processor -- start 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 -- start start 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.944+0000 7f54cf0fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c819ee00 0x7f54c819f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8114090 con 0x7f54c8102b50 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8113f10 con 0x7f54c819ee00 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f54c8114210 con 0x7f54c810e810 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40976/0 (socket says 172.21.15.129:40976) 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 -- 172.21.15.129:0/3425863862 learned_addr learned my addr 172.21.15.129:0/3425863862 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cd69a640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c819ee00 0x7f54c819f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:53.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54c7fff640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 -- 172.21.15.129:0/3425863862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 msgr2=0x7f54c819aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c819aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 -- 172.21.15.129:0/3425863862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c819ee00 msgr2=0x7f54c819f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c819ee00 0x7f54c819f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 -- 172.21.15.129:0/3425863862 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c819f8c0 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54c7fff640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cce99640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005940 tx=0x7f54bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc03f030 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54bc003cb0 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc035070 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.945+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c819fb50 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.946+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c81a72d0 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.947+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c8100090 con 0x7f54c8102b50 2024-09-05T01:12:53.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.948+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f54bc042030 con 0x7f54c8102b50 2024-09-05T01:12:53.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.948+0000 7f54c5ffb640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 0x7f54b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:53.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.948+0000 7f54c7fff640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 0x7f54b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:53.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.948+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f54bc0ef3b0 con 0x7f54c8102b50 2024-09-05T01:12:53.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.949+0000 7f54c7fff640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 0x7f54b80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54c80ff070 tx=0x7f54b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:53.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:53.952+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54bc0bc7b0 con 0x7f54c8102b50 2024-09-05T01:12:54.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.154+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54c8103150 con 0x7f54c8102b50 2024-09-05T01:12:54.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.154+0000 7f54c5ffb640 1 -- 172.21.15.129:0/3425863862 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9710 (secure 0 0 0) 0x7f54bc0bc0b0 con 0x7f54c8102b50 2024-09-05T01:12:54.155 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:54.155 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":12,"btime":"2024-09-05T01:12:53:831730+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-05T01:12:54.156 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 12 2024-09-05T01:12:54.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 msgr2=0x7f54b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:54.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 0x7f54b80a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54c80ff070 tx=0x7f54b4005f90 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 msgr2=0x7f54c819a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f54bc005940 tx=0x7f54bc0042f0 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 shutdown_connections 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54b80a2b80 0x7f54b80a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c819ee00 0x7f54c819f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f54c810e810 0x7f54c819aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 --2- 172.21.15.129:0/3425863862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54c8102b50 0x7f54c819a530 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 >> 172.21.15.129:0/3425863862 conn(0x7f54c8077210 msgr2=0x7f54c810fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 shutdown_connections 2024-09-05T01:12:54.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:54.157+0000 7f54cf0fd640 1 -- 172.21.15.129:0/3425863862 wait complete. 2024-09-05T01:12:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: Deploying daemon mds.b on smithi183 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.129:6840/3095518263,v1:172.21.15.129:6841/3095518263] up:boot 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: fsmap 11 up:standby 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T01:12:54.743 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:54 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3425863862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:54.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: Deploying daemon mds.b on smithi183 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.129:6840/3095518263,v1:172.21.15.129:6841/3095518263] up:boot 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: fsmap 11 up:standby 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T01:12:54.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:54 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3425863862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: Deploying daemon mds.b on smithi183 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.129:6840/3095518263,v1:172.21.15.129:6841/3095518263] up:boot 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: fsmap 11 up:standby 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T01:12:54.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:54 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3425863862' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:55.661 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:55 smithi183 systemd[1]: Starting Ceph mds.b for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:12:56.031 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:55 smithi183 podman[64597]: 2024-09-05 01:12:55.666062866 +0000 UTC m=+0.016428243 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:12:56.031 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:55 smithi183 podman[64597]: 2024-09-05 01:12:55.776811311 +0000 UTC m=+0.127176677 container create 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:12:56.031 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 podman[64597]: 2024-09-05 01:12:56.004580746 +0000 UTC m=+0.354946113 container init 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:12:56.032 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 podman[64597]: 2024-09-05 01:12:56.007540416 +0000 UTC m=+0.357905785 container start 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e) 2024-09-05T01:12:56.032 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mds[64616]: -- 172.21.15.183:0/2864683005 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f18de0b860 con 0x55f18dde8800 2024-09-05T01:12:56.410 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: starting mds.b at 2024-09-05T01:12:56.411 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 bash[64597]: 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 2024-09-05T01:12:56.411 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:12:56 smithi183 systemd[1]: Started Ceph mds.b for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:12:56.849 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.072 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:56 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.131 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:56 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:57.147 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:56 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:57.439 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:12:57.815 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:boot 2024-09-05T01:12:57.816 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: fsmap 12 up:standby 2024-09-05T01:12:57.816 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:12:57.816 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.816 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.816 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:57 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 -- 172.21.15.129:0/1957969346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c100e20 msgr2=0x7fa81c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 --2- 172.21.15.129:0/1957969346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c100e20 0x7fa81c101200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a480 comp rx=0 tx=0).stop 2024-09-05T01:12:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 -- 172.21.15.129:0/1957969346 shutdown_connections 2024-09-05T01:12:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 --2- 172.21.15.129:0/1957969346 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 0x7fa81c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:57.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 --2- 172.21.15.129:0/1957969346 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c102570 0x7fa81c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 --2- 172.21.15.129:0/1957969346 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c100e20 0x7fa81c101200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 -- 172.21.15.129:0/1957969346 >> 172.21.15.129:0/1957969346 conn(0x7fa81c0fc7d0 msgr2=0x7fa81c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 -- 172.21.15.129:0/1957969346 shutdown_connections 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.894+0000 7fa8226af640 1 -- 172.21.15.129:0/1957969346 wait complete. 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 Processor -- start 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 -- start start 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 0x7fa81c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 0x7fa81c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c110e80 con 0x7fa81c102570 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c110d00 con 0x7fa81c100e20 2024-09-05T01:12:57.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa8226af640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111000 con 0x7fa81c102ef0 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40992/0 (socket says 172.21.15.129:40992) 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa81b7fe640 1 -- 172.21.15.129:0/2621368763 learned_addr learned my addr 172.21.15.129:0/2621368763 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa81bfff640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 0x7fa81c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.895+0000 7fa820c4c640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 0x7fa81c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 -- 172.21.15.129:0/2621368763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 msgr2=0x7fa81c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 0x7fa81c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 -- 172.21.15.129:0/2621368763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 msgr2=0x7fa81c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 0x7fa81c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 -- 172.21.15.129:0/2621368763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c19d380 con 0x7fa81c102570 2024-09-05T01:12:57.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81bfff640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 0x7fa81c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:12:57.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa81b7fe640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa80400e9f0 tx=0x7fa80400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:57.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80400cd80 con 0x7fa81c102570 2024-09-05T01:12:57.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c19d670 con 0x7fa81c102570 2024-09-05T01:12:57.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8040108e0 con 0x7fa81c102570 2024-09-05T01:12:57.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804018920 con 0x7fa81c102570 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.896+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1a4f50 con 0x7fa81c102570 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.897+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c101200 con 0x7fa81c102570 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.898+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa804010a90 con 0x7fa81c102570 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.899+0000 7fa8197fa640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 0x7fa8080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.899+0000 7fa81bfff640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 0x7fa8080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:12:57.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.899+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa8040cf350 con 0x7fa81c102570 2024-09-05T01:12:57.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.899+0000 7fa81bfff640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 0x7fa8080a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c005e60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:12:57.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:57.902+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa80409c100 con 0x7fa81c102570 2024-09-05T01:12:58.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.118+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa81c10a0a0 con 0x7fa81c102570 2024-09-05T01:12:58.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.119+0000 7fa8197fa640 1 -- 172.21.15.129:0/2621368763 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10539 (secure 0 0 0) 0x7fa8040a0400 con 0x7fa81c102570 2024-09-05T01:12:58.120 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:12:58.120 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":13,"btime":"2024-09-05T01:12:56:795537+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-05T01:12:58.121 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 13 2024-09-05T01:12:58.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 msgr2=0x7fa8080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:58.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 0x7fa8080a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c005e60 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 msgr2=0x7fa81c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa80400e9f0 tx=0x7fa80400eeb0 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 shutdown_connections 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa8080a2b80 0x7fa8080a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa81c102ef0 0x7fa81c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa81c102570 0x7fa81c198910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 --2- 172.21.15.129:0/2621368763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa81c100e20 0x7fa81c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 >> 172.21.15.129:0/2621368763 conn(0x7fa81c0fc7d0 msgr2=0x7fa81c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 shutdown_connections 2024-09-05T01:12:58.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:12:58.123+0000 7fa8226af640 1 -- 172.21.15.129:0/2621368763 wait complete. 2024-09-05T01:12:58.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:boot 2024-09-05T01:12:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: fsmap 12 up:standby 2024-09-05T01:12:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:12:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:57 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:boot 2024-09-05T01:12:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: fsmap 12 up:standby 2024-09-05T01:12:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:12:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:57 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.845 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-05T01:12:58.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2621368763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:58.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:12:58 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2621368763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:59.137 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:59.138 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.138 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:12:58 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:59.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:12:59.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2621368763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:12:59.221 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:12:59.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:12:59.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:12:59.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:12:58 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:12:59.772 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:00.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 -- 172.21.15.129:0/3300089851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 msgr2=0x7f59501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 --2- 172.21.15.129:0/3300089851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59501021c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f594c009900 tx=0x7f594c02a410 comp rx=0 tx=0).stop 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 -- 172.21.15.129:0/3300089851 shutdown_connections 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 --2- 172.21.15.129:0/3300089851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950106f10 0x7f595010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 --2- 172.21.15.129:0/3300089851 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950102790 0x7f59501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 --2- 172.21.15.129:0/3300089851 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59501021c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 -- 172.21.15.129:0/3300089851 >> 172.21.15.129:0/3300089851 conn(0x7f59500fd8c0 msgr2=0x7f59500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:00.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 -- 172.21.15.129:0/3300089851 shutdown_connections 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.231+0000 7f5958128640 1 -- 172.21.15.129:0/3300089851 wait complete. 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 Processor -- start 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 -- start start 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950102790 0x7f5950074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950106f10 0x7f5950074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111de0 con 0x7f5950101de0 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111c60 con 0x7f5950102790 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5958128640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111f60 con 0x7f5950106f10 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41006/0 (socket says 172.21.15.129:41006) 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 -- 172.21.15.129:0/4017079110 learned_addr learned my addr 172.21.15.129:0/4017079110 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:00.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f59556c3640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950102790 0x7f5950074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f59566c5640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950106f10 0x7f5950074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 -- 172.21.15.129:0/4017079110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950106f10 msgr2=0x7f5950074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950106f10 0x7f5950074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 -- 172.21.15.129:0/4017079110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950102790 msgr2=0x7f5950074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950102790 0x7f5950074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.232+0000 7f5955ec4640 1 -- 172.21.15.129:0/4017079110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59501a6880 con 0x7f5950101de0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5955ec4640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f594c005940 tx=0x7f594c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c03f030 con 0x7f5950101de0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59501a6b10 con 0x7f5950101de0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f594c003aa0 con 0x7f5950101de0 2024-09-05T01:13:00.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c035070 con 0x7f5950101de0 2024-09-05T01:13:00.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.233+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501a6ff0 con 0x7f5950101de0 2024-09-05T01:13:00.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.234+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59501021c0 con 0x7f5950101de0 2024-09-05T01:13:00.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.236+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f594c042030 con 0x7f5950101de0 2024-09-05T01:13:00.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.236+0000 7f5946ffd640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 0x7f593c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:00.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.237+0000 7f59556c3640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 0x7f593c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:00.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.237+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f594c0ef500 con 0x7f5950101de0 2024-09-05T01:13:00.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.237+0000 7f59556c3640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 0x7f593c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5950075f80 tx=0x7f594000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:00.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.240+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f594c0bc2b0 con 0x7f5950101de0 2024-09-05T01:13:00.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.431+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5950075a20 con 0x7f5950101de0 2024-09-05T01:13:00.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.431+0000 7f5946ffd640 1 -- 172.21.15.129:0/4017079110 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f594c0b6e20 con 0x7f5950101de0 2024-09-05T01:13:00.432 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:00.432 INFO:teuthology.orchestra.run.smithi129.stdout:[ 2024-09-05T01:13:00.433 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:13:00.433 INFO:teuthology.orchestra.run.smithi129.stdout: "poolnum": 1, 2024-09-05T01:13:00.433 INFO:teuthology.orchestra.run.smithi129.stdout: "poolname": ".mgr" 2024-09-05T01:13:00.433 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:13:00.433 INFO:teuthology.orchestra.run.smithi129.stdout:] 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 msgr2=0x7f593c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 0x7f593c0a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5950075f80 tx=0x7f594000b410 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 msgr2=0x7f59500744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f594c005940 tx=0x7f594c004270 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 shutdown_connections 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f593c0a2b80 0x7f593c0a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5950106f10 0x7f5950074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5950102790 0x7f5950074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 --2- 172.21.15.129:0/4017079110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5950101de0 0x7f59500744f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 >> 172.21.15.129:0/4017079110 conn(0x7f59500fd8c0 msgr2=0x7f59500fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:00.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 shutdown_connections 2024-09-05T01:13:00.437 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:00.435+0000 7f5958128640 1 -- 172.21.15.129:0/4017079110 wait complete. 2024-09-05T01:13:01.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:01.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4017079110' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:13:01.116 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.143 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-05T01:13:01.143 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-05T01:13:01.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4017079110' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:13:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4017079110' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:13:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:13:01.388 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:01.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1522679616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0106f10 msgr2=0x7fc9d010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:01.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1522679616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0106f10 0x7fc9d010ea90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009ca0 tx=0x7fc9c802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1522679616 shutdown_connections 2024-09-05T01:13:01.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1522679616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0106f10 0x7fc9d010ea90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1522679616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d0102790 0x7fc9d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1522679616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0101de0 0x7fc9d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1522679616 >> 172.21.15.129:0/1522679616 conn(0x7fc9d00fd8c0 msgr2=0x7fc9d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1522679616 shutdown_connections 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.818+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1522679616 wait complete. 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 Processor -- start 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 -- start start 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 0x7fc9d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d0106f10 0x7fc9d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111e20 con 0x7fc9d0101de0 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111ca0 con 0x7fc9d0106f10 2024-09-05T01:13:01.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d6d78640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0111fa0 con 0x7fc9d0102790 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d4b14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d4b14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41022/0 (socket says 172.21.15.129:41022) 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.819+0000 7fc9d4b14640 1 -- 172.21.15.129:0/1102538806 learned_addr learned my addr 172.21.15.129:0/1102538806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9cffff640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 0x7fc9d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 -- 172.21.15.129:0/1102538806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 msgr2=0x7fc9d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 0x7fc9d0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 -- 172.21.15.129:0/1102538806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d0106f10 msgr2=0x7fc9d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d0106f10 0x7fc9d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 -- 172.21.15.129:0/1102538806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d01a6800 con 0x7fc9d0101de0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9cffff640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 0x7fc9d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d4b14640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400d770 tx=0x7fc9c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4017070 con 0x7fc9d0101de0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9c40054d0 con 0x7fc9d0101de0 2024-09-05T01:13:01.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d01a6af0 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4004cf0 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.820+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a6fe0 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.821+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9d01021c0 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.822+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc9c4004e90 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.823+0000 7fc9cdffb640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 0x7fc9c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.823+0000 7fc9cffff640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 0x7fc9c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.823+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc9c40ce6c0 con 0x7fc9d0101de0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.823+0000 7fc9cffff640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 0x7fc9c00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0075f80 tx=0x7fc9bc005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:01.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.827+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9c409bac0 con 0x7fc9d0101de0 2024-09-05T01:13:01.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:01.995+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc9d002d7c0 con 0x7fc9d0101de0 2024-09-05T01:13:02.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.865+0000 7fc9cdffb640 1 -- 172.21.15.129:0/1102538806 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fc9c409b3c0 con 0x7fc9d0101de0 2024-09-05T01:13:02.866 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' created 2024-09-05T01:13:02.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.867+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 msgr2=0x7fc9c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:02.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.867+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 0x7fc9c00a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0075f80 tx=0x7fc9bc005c90 comp rx=0 tx=0).stop 2024-09-05T01:13:02.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.867+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 msgr2=0x7fc9d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.867+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400d770 tx=0x7fc9c400dc30 comp rx=0 tx=0).stop 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 shutdown_connections 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9c00a2b80 0x7fc9c00a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9d0106f10 0x7fc9d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc9d0102790 0x7fc9d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 --2- 172.21.15.129:0/1102538806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9d0101de0 0x7fc9d00744f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 >> 172.21.15.129:0/1102538806 conn(0x7fc9d00fd8c0 msgr2=0x7fc9d00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 shutdown_connections 2024-09-05T01:13:02.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:02.868+0000 7fc9d6d78640 1 -- 172.21.15.129:0/1102538806 wait complete. 2024-09-05T01:13:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:02 smithi183 ceph-mon[34589]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:02 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T01:13:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:02 smithi129 ceph-mon[30979]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:02 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T01:13:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:02 smithi194 ceph-mon[34941]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:02 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T01:13:03.589 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-05T01:13:03.840 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:03 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T01:13:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:03 smithi183 ceph-mon[34589]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T01:13:04.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:03 smithi183 ceph-mon[34589]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T01:13:04.229 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:03 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T01:13:04.229 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:03 smithi129 ceph-mon[30979]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T01:13:04.230 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:03 smithi129 ceph-mon[30979]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T01:13:04.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:03 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1102538806' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T01:13:04.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:03 smithi194 ceph-mon[34941]: osdmap e76: 12 total, 12 up, 12 in 2024-09-05T01:13:04.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:03 smithi194 ceph-mon[34941]: osdmap e77: 12 total, 12 up, 12 in 2024-09-05T01:13:04.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 -- 172.21.15.129:0/2835181992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0102790 msgr2=0x7f58e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 --2- 172.21.15.129:0/2835181992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0102790 0x7f58e01067e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009900 tx=0x7f58d002a410 comp rx=0 tx=0).stop 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 -- 172.21.15.129:0/2835181992 shutdown_connections 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 --2- 172.21.15.129:0/2835181992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 --2- 172.21.15.129:0/2835181992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0102790 0x7f58e01067e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 --2- 172.21.15.129:0/2835181992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101de0 0x7f58e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 -- 172.21.15.129:0/2835181992 >> 172.21.15.129:0/2835181992 conn(0x7f58e00fd8c0 msgr2=0x7f58e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 -- 172.21.15.129:0/2835181992 shutdown_connections 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.315+0000 7f58e7445640 1 -- 172.21.15.129:0/2835181992 wait complete. 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 Processor -- start 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 -- start start 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:04.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0102790 0x7f58e0198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0111ec0 con 0x7f58e0101de0 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0111d40 con 0x7f58e0106f10 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e7445640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0112040 con 0x7f58e0102790 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e51e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e51e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41242/0 (socket says 172.21.15.129:41242) 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e51e1640 1 -- 172.21.15.129:0/504782305 learned_addr learned my addr 172.21.15.129:0/504782305 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e59e2640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.316+0000 7f58e49e0640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0102790 0x7f58e0198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 -- 172.21.15.129:0/504782305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0102790 msgr2=0x7f58e0198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0102790 0x7f58e0198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 -- 172.21.15.129:0/504782305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 msgr2=0x7f58e019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 -- 172.21.15.129:0/504782305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e019d490 con 0x7f58e0101de0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e59e2640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e51e1640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b4a0 tx=0x7f58dc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc014070 con 0x7f58e0101de0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58dc010ac0 con 0x7f58e0101de0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e019d780 con 0x7f58e0101de0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc017c30 con 0x7f58e0101de0 2024-09-05T01:13:04.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.317+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e01a5060 con 0x7f58e0101de0 2024-09-05T01:13:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.318+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e0102240 con 0x7f58e0101de0 2024-09-05T01:13:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.320+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f58dc017070 con 0x7f58e0101de0 2024-09-05T01:13:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.320+0000 7f58d67fc640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 0x7f58cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:04.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.321+0000 7f58e49e0640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 0x7f58cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.321+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7f58dc0ce570 con 0x7f58e0101de0 2024-09-05T01:13:04.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.321+0000 7f58e49e0640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 0x7f58cc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58d0031440 tx=0x7f58d00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:04.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.324+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58dc09b800 con 0x7f58e0101de0 2024-09-05T01:13:04.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.489+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f58e010a150 con 0x7f58e0101de0 2024-09-05T01:13:04.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.868+0000 7f58d67fc640 1 -- 172.21.15.129:0/504782305 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) ==== 171+0+0 (secure 0 0 0) 0x7f58dc09b100 con 0x7f58e0101de0 2024-09-05T01:13:04.869 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' created 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 msgr2=0x7f58cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 0x7f58cc0a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58d0031440 tx=0x7f58d00095f0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 msgr2=0x7f58e01984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b4a0 tx=0x7f58dc00b960 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 shutdown_connections 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2b80 0x7f58cc0a5040 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f58d0031440 tx=0x7f58d00095f0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106f10 0x7f58e019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0102790 0x7f58e0198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 --2- 172.21.15.129:0/504782305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0101de0 0x7f58e01984e0 secure :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b4a0 tx=0x7f58dc00b960 comp rx=0 tx=0).stop 2024-09-05T01:13:04.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.872+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 >> 172.21.15.129:0/504782305 conn(0x7f58e00fd8c0 msgr2=0x7f58e00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:04.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.873+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 shutdown_connections 2024-09-05T01:13:04.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:04.873+0000 7f58e7445640 1 -- 172.21.15.129:0/504782305 wait complete. 2024-09-05T01:13:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:04 smithi183 ceph-mon[34589]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:04 smithi183 ceph-mon[34589]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:04 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T01:13:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:04 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T01:13:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:04 smithi183 ceph-mon[34589]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T01:13:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:04 smithi129 ceph-mon[30979]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:04 smithi129 ceph-mon[30979]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:04 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T01:13:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:04 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T01:13:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:04 smithi129 ceph-mon[30979]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T01:13:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:04 smithi194 ceph-mon[34941]: pgmap v269: 33 pgs: 3 creating+peering, 29 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:04 smithi194 ceph-mon[34941]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:04 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T01:13:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:04 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/504782305' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T01:13:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:04 smithi194 ceph-mon[34941]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T01:13:05.537 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-05T01:13:05.784 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:06.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 -- 172.21.15.129:0/3994296885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8100a80 msgr2=0x7f73b8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 --2- 172.21.15.129:0/3994296885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8100a80 0x7f73b8100ee0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73a4009930 tx=0x7f73a402a420 comp rx=0 tx=0).stop 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 -- 172.21.15.129:0/3994296885 shutdown_connections 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 --2- 172.21.15.129:0/3994296885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8101420 0x7f73b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 --2- 172.21.15.129:0/3994296885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8100a80 0x7f73b8100ee0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 --2- 172.21.15.129:0/3994296885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8106a80 0x7f73b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 -- 172.21.15.129:0/3994296885 >> 172.21.15.129:0/3994296885 conn(0x7f73b80fc7a0 msgr2=0x7f73b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 -- 172.21.15.129:0/3994296885 shutdown_connections 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 -- 172.21.15.129:0/3994296885 wait complete. 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.286+0000 7f73bef69640 1 Processor -- start 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 -- start start 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8100a80 0x7f73b80fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:06.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 0x7f73b80fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8110dd0 con 0x7f73b8101420 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8110c50 con 0x7f73b8106a80 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73b8110f50 con 0x7f73b8100a80 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41258/0 (socket says 172.21.15.129:41258) 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 -- 172.21.15.129:0/3673531268 learned_addr learned my addr 172.21.15.129:0/3673531268 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bcd05640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8100a80 0x7f73b80fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bd506640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 0x7f73b80fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 -- 172.21.15.129:0/3673531268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8100a80 msgr2=0x7f73b80fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:06.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8100a80 0x7f73b80fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 -- 172.21.15.129:0/3673531268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 msgr2=0x7f73b80fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 0x7f73b80fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 -- 172.21.15.129:0/3673531268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b81a69d0 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bd506640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 0x7f73b80fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b7fff640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73a4005940 tx=0x7f73a40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a403f030 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73a4003cb0 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73b81a6c60 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.287+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a4035070 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.288+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73b81a7190 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.289+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73b8100f60 con 0x7f73b8101420 2024-09-05T01:13:06.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.290+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f73a4042030 con 0x7f73b8101420 2024-09-05T01:13:06.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.291+0000 7f73b5ffb640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 0x7f73a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:06.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.291+0000 7f73bcd05640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 0x7f73a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:06.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.291+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f73a40ef740 con 0x7f73b8101420 2024-09-05T01:13:06.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.291+0000 7f73bcd05640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 0x7f73a80a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009890 tx=0x7f73ac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:06.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.295+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73a40bc7b0 con 0x7f73b8101420 2024-09-05T01:13:06.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.499+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f73b80fe3a0 con 0x7f73b8101420 2024-09-05T01:13:06.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.884+0000 7f73b5ffb640 1 -- 172.21.15.129:0/3673531268 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f73a40bc0b0 con 0x7f73b8101420 2024-09-05T01:13:06.885 INFO:teuthology.orchestra.run.smithi129.stderr:new fs with metadata pool 2 and data pool 3 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 msgr2=0x7f73a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 0x7f73a80a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009890 tx=0x7f73ac008040 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 msgr2=0x7f73b80fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f73a4005940 tx=0x7f73a40042f0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 shutdown_connections 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73a80a2c00 0x7f73a80a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73b8106a80 0x7f73b80fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73b8101420 0x7f73b80fd3b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 --2- 172.21.15.129:0/3673531268 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73b8100a80 0x7f73b80fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:06.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 >> 172.21.15.129:0/3673531268 conn(0x7f73b80fc7a0 msgr2=0x7f73b806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:06.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 shutdown_connections 2024-09-05T01:13:06.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:06.887+0000 7f73bef69640 1 -- 172.21.15.129:0/3673531268 wait complete. 2024-09-05T01:13:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:06 smithi183 ceph-mon[34589]: pgmap v272: 97 pgs: 13 creating+peering, 82 unknown, 2 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:06 smithi183 ceph-mon[34589]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T01:13:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:06 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T01:13:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:06 smithi183 ceph-mon[34589]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T01:13:07.226 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:06 smithi129 ceph-mon[30979]: pgmap v272: 97 pgs: 13 creating+peering, 82 unknown, 2 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:07.226 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:06 smithi129 ceph-mon[30979]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T01:13:07.226 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:06 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T01:13:07.226 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:06 smithi129 ceph-mon[30979]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T01:13:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:06 smithi194 ceph-mon[34941]: pgmap v272: 97 pgs: 13 creating+peering, 82 unknown, 2 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:06 smithi194 ceph-mon[34941]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T01:13:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:06 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T01:13:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:06 smithi194 ceph-mon[34941]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T01:13:07.508 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-05T01:13:07.508 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-05T01:13:07.756 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:08.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:07 smithi129 ceph-mon[30979]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T01:13:08.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:07 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T01:13:08.139 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:07 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T01:13:08.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:07 smithi129 ceph-mon[30979]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T01:13:08.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:07 smithi183 ceph-mon[34589]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T01:13:08.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:07 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T01:13:08.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:07 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T01:13:08.158 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:07 smithi183 ceph-mon[34589]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 -- 172.21.15.129:0/2552809810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85441082a0 msgr2=0x7f854410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 --2- 172.21.15.129:0/2552809810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85441082a0 0x7f854410ec40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f853c009930 tx=0x7f853c02a420 comp rx=0 tx=0).stop 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 -- 172.21.15.129:0/2552809810 shutdown_connections 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 --2- 172.21.15.129:0/2552809810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f85441082a0 0x7f854410ec40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 --2- 172.21.15.129:0/2552809810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441036e0 0x7f8544107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 --2- 172.21.15.129:0/2552809810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8544102d30 0x7f8544103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.223+0000 7f85499d1640 1 -- 172.21.15.129:0/2552809810 >> 172.21.15.129:0/2552809810 conn(0x7f85440fc7d0 msgr2=0x7f85440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.224+0000 7f85499d1640 1 -- 172.21.15.129:0/2552809810 shutdown_connections 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.224+0000 7f85499d1640 1 -- 172.21.15.129:0/2552809810 wait complete. 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.224+0000 7f85499d1640 1 Processor -- start 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 -- start start 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85441036e0 0x7f8544198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441082a0 0x7f854419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8544111fb0 con 0x7f8544102d30 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8544111e30 con 0x7f85441082a0 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8544112130 con 0x7f85441036e0 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:08.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41272/0 (socket says 172.21.15.129:41272) 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 -- 172.21.15.129:0/3695552200 learned_addr learned my addr 172.21.15.129:0/3695552200 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85437fe640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441082a0 0x7f854419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 -- 172.21.15.129:0/3695552200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85441036e0 msgr2=0x7f8544198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85441036e0 0x7f8544198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 -- 172.21.15.129:0/3695552200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441082a0 msgr2=0x7f854419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441082a0 0x7f854419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 -- 172.21.15.129:0/3695552200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854419d4b0 con 0x7f8544102d30 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f8542ffd640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f853400ba50 tx=0x7f853400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534019070 con 0x7f8544102d30 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854419d7a0 con 0x7f8544102d30 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f853400cbe0 con 0x7f8544102d30 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.225+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340075d0 con 0x7f8544102d30 2024-09-05T01:13:08.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.226+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85441a5080 con 0x7f8544102d30 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.227+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854406a850 con 0x7f8544102d30 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.228+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f853401c070 con 0x7f8544102d30 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.228+0000 7f85489cf640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 0x7f852c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.229+0000 7f85427fc640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 0x7f852c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.229+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f85340ce5d0 con 0x7f8544102d30 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.229+0000 7f85427fc640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 0x7f852c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8538005fd0 tx=0x7f8538005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:08.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.233+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f853409b680 con 0x7f8544102d30 2024-09-05T01:13:08.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:07 smithi194 ceph-mon[34941]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T01:13:08.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:07 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3673531268' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T01:13:08.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:07 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T01:13:08.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:07 smithi194 ceph-mon[34941]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T01:13:08.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.406+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f8544103110 con 0x7f8544102d30 2024-09-05T01:13:08.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.891+0000 7f85489cf640 1 -- 172.21.15.129:0/3695552200 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f853409af80 con 0x7f8544102d30 2024-09-05T01:13:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.893+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 msgr2=0x7f852c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.893+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 0x7f852c0a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8538005fd0 tx=0x7f8538005850 comp rx=0 tx=0).stop 2024-09-05T01:13:08.894 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.893+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 msgr2=0x7f8544198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.893+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f853400ba50 tx=0x7f853400bf10 comp rx=0 tx=0).stop 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 shutdown_connections 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f852c0a2b80 0x7f852c0a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85441082a0 0x7f854419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f85441036e0 0x7f8544198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 --2- 172.21.15.129:0/3695552200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8544102d30 0x7f8544198500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 >> 172.21.15.129:0/3695552200 conn(0x7f85440fc7d0 msgr2=0x7f8544105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 shutdown_connections 2024-09-05T01:13:08.895 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:08.894+0000 7f85499d1640 1 -- 172.21.15.129:0/3695552200 wait complete. 2024-09-05T01:13:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: pgmap v275: 97 pgs: 13 creating+peering, 46 unknown, 38 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:active 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:08 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: pgmap v275: 97 pgs: 13 creating+peering, 46 unknown, 38 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:active 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:08 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T01:13:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: pgmap v275: 97 pgs: 13 creating+peering, 46 unknown, 38 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T01:13:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:13:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] up:active 2024-09-05T01:13:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:09.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:08 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T01:13:09.531 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-05T01:13:09.776 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:10.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:09 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T01:13:10.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:09 smithi183 ceph-mon[34589]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T01:13:10.174 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:09 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T01:13:10.174 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:09 smithi129 ceph-mon[30979]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T01:13:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:09 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3695552200' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T01:13:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:09 smithi194 ceph-mon[34941]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T01:13:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.261+0000 7fab106ad640 1 -- 172.21.15.129:0/616360787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08102570 msgr2=0x7fab081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.261+0000 7fab106ad640 1 --2- 172.21.15.129:0/616360787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08102570 0x7fab081029b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009900 tx=0x7faaf802a480 comp rx=0 tx=0).stop 2024-09-05T01:13:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- 172.21.15.129:0/616360787 shutdown_connections 2024-09-05T01:13:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- 172.21.15.129:0/616360787 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102ef0 0x7fab0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:10.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- 172.21.15.129:0/616360787 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08102570 0x7fab081029b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- 172.21.15.129:0/616360787 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08100e20 0x7fab08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- 172.21.15.129:0/616360787 >> 172.21.15.129:0/616360787 conn(0x7fab080fc7d0 msgr2=0x7fab080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- 172.21.15.129:0/616360787 shutdown_connections 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- 172.21.15.129:0/616360787 wait complete. 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 Processor -- start 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- start start 2024-09-05T01:13:10.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:10.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 0x7fab081989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:10.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 0x7fab0819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:10.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab08110e70 con 0x7fab08100e20 2024-09-05T01:13:10.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fab08110cf0 con 0x7fab08102570 2024-09-05T01:13:10.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.262+0000 7fab106ad640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fab08110ff0 con 0x7fab08102ef0 2024-09-05T01:13:10.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:10.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41304/0 (socket says 172.21.15.129:41304) 2024-09-05T01:13:10.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 -- 172.21.15.129:0/1107859572 learned_addr learned my addr 172.21.15.129:0/1107859572 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:10.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0dc48640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 0x7fab081989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:10.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0ec4a640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 0x7fab0819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:10.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 -- 172.21.15.129:0/1107859572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 msgr2=0x7fab0819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:10.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 0x7fab0819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:10.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 -- 172.21.15.129:0/1107859572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 msgr2=0x7fab081989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:10.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 0x7fab081989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:10.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 -- 172.21.15.129:0/1107859572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0819d440 con 0x7fab08100e20 2024-09-05T01:13:10.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0ec4a640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 0x7fab0819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:10.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0dc48640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 0x7fab081989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:10.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab0e449640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab0400c970 tx=0x7fab0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:10.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab04007bb0 con 0x7fab08100e20 2024-09-05T01:13:10.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0819d730 con 0x7fab08100e20 2024-09-05T01:13:10.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab04007d50 con 0x7fab08100e20 2024-09-05T01:13:10.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0400f450 con 0x7fab08100e20 2024-09-05T01:13:10.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.263+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab080fcbb0 con 0x7fab08100e20 2024-09-05T01:13:10.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.264+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08101200 con 0x7fab08100e20 2024-09-05T01:13:10.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.265+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fab04004f60 con 0x7fab08100e20 2024-09-05T01:13:10.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.266+0000 7faaff7fe640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 0x7faaf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:10.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.266+0000 7fab0dc48640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 0x7faaf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:10.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.266+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7fab040ceec0 con 0x7fab08100e20 2024-09-05T01:13:10.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.267+0000 7fab0dc48640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 0x7faaf40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab080ff0e0 tx=0x7faaf8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:10.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.270+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab04014030 con 0x7fab08100e20 2024-09-05T01:13:10.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:10.437+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fab0810a0a0 con 0x7fab08100e20 2024-09-05T01:13:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:11 smithi183 ceph-mon[34589]: pgmap v277: 97 pgs: 7 creating+peering, 6 unknown, 84 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-09-05T01:13:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:11 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T01:13:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:11 smithi129 ceph-mon[30979]: pgmap v277: 97 pgs: 7 creating+peering, 6 unknown, 84 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-09-05T01:13:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:11 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T01:13:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:11 smithi194 ceph-mon[34941]: pgmap v277: 97 pgs: 7 creating+peering, 6 unknown, 84 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 4 op/s 2024-09-05T01:13:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:11 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T01:13:11.900 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.900+0000 7faaff7fe640 1 -- 172.21.15.129:0/1107859572 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7fab0409bdf0 con 0x7fab08100e20 2024-09-05T01:13:11.901 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data_ec' created 2024-09-05T01:13:11.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 msgr2=0x7faaf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:11.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 0x7faaf40a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fab080ff0e0 tx=0x7faaf8005a20 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 msgr2=0x7fab08198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab0400c970 tx=0x7fab0400ce30 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 shutdown_connections 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faaf40a2b80 0x7faaf40a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fab08102ef0 0x7fab0819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fab08102570 0x7fab081989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 --2- 172.21.15.129:0/1107859572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab08100e20 0x7fab08198490 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 >> 172.21.15.129:0/1107859572 conn(0x7fab080fc7d0 msgr2=0x7fab0806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:11.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 shutdown_connections 2024-09-05T01:13:11.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:11.903+0000 7fab106ad640 1 -- 172.21.15.129:0/1107859572 wait complete. 2024-09-05T01:13:12.136 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:13:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: 2024-09-05T01:13:11.803+0000 7f8eb1ae0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-05T01:13:12.202 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:12 smithi129 ceph-mon[30979]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T01:13:12.203 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:12 smithi129 ceph-mon[30979]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-05T01:13:12.203 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:12 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T01:13:12.203 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:12 smithi129 ceph-mon[30979]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T01:13:12.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:12 smithi183 ceph-mon[34589]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T01:13:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:12 smithi183 ceph-mon[34589]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-05T01:13:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:12 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T01:13:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:12 smithi183 ceph-mon[34589]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T01:13:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:12 smithi194 ceph-mon[34941]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T01:13:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:12 smithi194 ceph-mon[34941]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-09-05T01:13:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:12 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1107859572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T01:13:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:12 smithi194 ceph-mon[34941]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T01:13:12.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-05T01:13:12.750 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:13.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.209+0000 7fc69b771640 1 -- 172.21.15.129:0/1395272273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694102ef0 msgr2=0x7fc69410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.209+0000 7fc69b771640 1 --2- 172.21.15.129:0/1395272273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694102ef0 0x7fc69410da40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009ca0 tx=0x7fc68c02a950 comp rx=0 tx=0).stop 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 -- 172.21.15.129:0/1395272273 shutdown_connections 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 --2- 172.21.15.129:0/1395272273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694102ef0 0x7fc69410da40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 --2- 172.21.15.129:0/1395272273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 0x7fc6941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 --2- 172.21.15.129:0/1395272273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694100e20 0x7fc694101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 -- 172.21.15.129:0/1395272273 >> 172.21.15.129:0/1395272273 conn(0x7fc6940fc7d0 msgr2=0x7fc6940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 -- 172.21.15.129:0/1395272273 shutdown_connections 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 -- 172.21.15.129:0/1395272273 wait complete. 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 Processor -- start 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 -- start start 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.210+0000 7fc69b771640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:13.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 0x7fc694198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694102ef0 0x7fc69419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc694110e30 con 0x7fc694100e20 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc694110cb0 con 0x7fc694102570 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc694110fb0 con 0x7fc694102ef0 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43030/0 (socket says 172.21.15.129:43030) 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 -- 172.21.15.129:0/1362269233 learned_addr learned my addr 172.21.15.129:0/1362269233 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc698d0c640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 0x7fc694198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc699d0e640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694102ef0 0x7fc69419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 -- 172.21.15.129:0/1362269233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694102ef0 msgr2=0x7fc69419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694102ef0 0x7fc69419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 -- 172.21.15.129:0/1362269233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 msgr2=0x7fc694198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:13.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 0x7fc694198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 -- 172.21.15.129:0/1362269233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69419d2f0 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69950d640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc69000d950 tx=0x7fc69000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690017070 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69419d5e0 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc690009c30 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc69000f620 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.211+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6941a4ec0 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.212+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc694101200 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.214+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc6900109a0 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.215+0000 7fc68a7fc640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 0x7fc6800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.215+0000 7fc698d0c640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 0x7fc6800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.215+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7fc6900cedc0 con 0x7fc694100e20 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.215+0000 7fc698d0c640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 0x7fc6800a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ff060 tx=0x7fc68400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:13.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.219+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc690009de0 con 0x7fc694100e20 2024-09-05T01:13:13.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.390+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc69410a0a0 con 0x7fc694100e20 2024-09-05T01:13:13.907 INFO:teuthology.orchestra.run.smithi129.stderr:set pool 4 allow_ec_overwrites to true 2024-09-05T01:13:13.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.906+0000 7fc68a7fc640 1 -- 172.21.15.129:0/1362269233 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7fc69009b4b0 con 0x7fc694100e20 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 msgr2=0x7fc6800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 0x7fc6800a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc6940ff060 tx=0x7fc68400a2d0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 msgr2=0x7fc694198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc69000d950 tx=0x7fc69000de10 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 shutdown_connections 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc6800a2b80 0x7fc6800a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc694102ef0 0x7fc69419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc694102570 0x7fc694198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 --2- 172.21.15.129:0/1362269233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc694100e20 0x7fc694198340 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:13.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 >> 172.21.15.129:0/1362269233 conn(0x7fc6940fc7d0 msgr2=0x7fc69406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 shutdown_connections 2024-09-05T01:13:13.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:13.909+0000 7fc69b771640 1 -- 172.21.15.129:0/1362269233 wait complete. 2024-09-05T01:13:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:13 smithi183 ceph-mon[34589]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T01:13:14.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:13 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T01:13:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:13 smithi129 ceph-mon[30979]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T01:13:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:13 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T01:13:14.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:13 smithi194 ceph-mon[34941]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T01:13:14.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:13 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T01:13:14.658 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-05T01:13:14.910 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:15.158 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:14 smithi129 ceph-mon[30979]: pgmap v282: 129 pgs: 8 creating+peering, 21 unknown, 100 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 210 B/s wr, 4 op/s 2024-09-05T01:13:15.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:14 smithi129 ceph-mon[30979]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:15.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:14 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T01:13:15.159 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:14 smithi129 ceph-mon[30979]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T01:13:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:14 smithi183 ceph-mon[34589]: pgmap v282: 129 pgs: 8 creating+peering, 21 unknown, 100 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 210 B/s wr, 4 op/s 2024-09-05T01:13:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:14 smithi183 ceph-mon[34589]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:14 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T01:13:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:14 smithi183 ceph-mon[34589]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T01:13:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:14 smithi194 ceph-mon[34941]: pgmap v282: 129 pgs: 8 creating+peering, 21 unknown, 100 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 210 B/s wr, 4 op/s 2024-09-05T01:13:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:14 smithi194 ceph-mon[34941]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T01:13:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:14 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1362269233' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T01:13:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:14 smithi194 ceph-mon[34941]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T01:13:15.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 -- 172.21.15.129:0/580811647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b10103660 msgr2=0x7f8b1010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 --2- 172.21.15.129:0/580811647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b10103660 0x7f8b1010bb50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009900 tx=0x7f8af802a480 comp rx=0 tx=0).stop 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 -- 172.21.15.129:0/580811647 shutdown_connections 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 --2- 172.21.15.129:0/580811647 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b1010c170 0x7f8b1010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 --2- 172.21.15.129:0/580811647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b10103660 0x7f8b1010bb50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 --2- 172.21.15.129:0/580811647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10102d40 0x7f8b10103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 -- 172.21.15.129:0/580811647 >> 172.21.15.129:0/580811647 conn(0x7f8b100fc7a0 msgr2=0x7f8b100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 -- 172.21.15.129:0/580811647 shutdown_connections 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.358+0000 7f8b1544d640 1 -- 172.21.15.129:0/580811647 wait complete. 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 Processor -- start 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 -- start start 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b10102d40 0x7f8b10198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10103660 0x7f8b10198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10111e20 con 0x7f8b1010c170 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10111ca0 con 0x7f8b10103660 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b1544d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8b10111fa0 con 0x7f8b10102d40 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b0f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43042/0 (socket says 172.21.15.129:43042) 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b0f7fe640 1 -- 172.21.15.129:0/928568922 learned_addr learned my addr 172.21.15.129:0/928568922 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.359+0000 7f8b0effd640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b10102d40 0x7f8b10198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0e7fc640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10103660 0x7f8b10198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 -- 172.21.15.129:0/928568922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b10102d40 msgr2=0x7f8b10198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b10102d40 0x7f8b10198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 -- 172.21.15.129:0/928568922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10103660 msgr2=0x7f8b10198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10103660 0x7f8b10198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 -- 172.21.15.129:0/928568922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1019d2d0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b0f7fe640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000eaa0 tx=0x7f8b0000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00004d20 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b00005020 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1019d5c0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b000116b0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.360+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b101a4ea0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.361+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b10103120 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.362+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8b000051d0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.363+0000 7f8b07fff640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 0x7f8afc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.363+0000 7f8b0effd640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 0x7f8afc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.363+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f8b000cf3d0 con 0x7f8b1010c170 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.363+0000 7f8b0effd640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 0x7f8afc0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800abe0 tx=0x7f8b0800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.367+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b00016030 con 0x7f8b1010c170 2024-09-05T01:13:15.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.570+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f8b1002d7c0 con 0x7f8b1010c170 2024-09-05T01:13:15.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.914+0000 7f8b07fff640 1 -- 172.21.15.129:0/928568922 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f8b0009c110 con 0x7f8b1010c170 2024-09-05T01:13:15.915 INFO:teuthology.orchestra.run.smithi129.stderr:added data pool 4 to fsmap 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 msgr2=0x7f8afc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 0x7f8afc0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800abe0 tx=0x7f8b0800a2d0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 msgr2=0x7f8b1019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b0000eaa0 tx=0x7f8b0000eea0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 shutdown_connections 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8afc0a2b80 0x7f8afc0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b1010c170 0x7f8b1019cbf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8b10103660 0x7f8b10198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 --2- 172.21.15.129:0/928568922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8b10102d40 0x7f8b10198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 >> 172.21.15.129:0/928568922 conn(0x7f8b100fc7a0 msgr2=0x7f8b100fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 shutdown_connections 2024-09-05T01:13:15.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:15.917+0000 7f8b1544d640 1 -- 172.21.15.129:0/928568922 wait complete. 2024-09-05T01:13:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:15 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T01:13:16.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:15 smithi183 ceph-mon[34589]: pgmap v284: 129 pgs: 8 creating+peering, 11 unknown, 110 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-05T01:13:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:15 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T01:13:16.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:15 smithi129 ceph-mon[30979]: pgmap v284: 129 pgs: 8 creating+peering, 11 unknown, 110 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-05T01:13:16.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:15 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T01:13:16.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:15 smithi194 ceph-mon[34941]: pgmap v284: 129 pgs: 8 creating+peering, 11 unknown, 110 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 211 B/s wr, 4 op/s 2024-09-05T01:13:16.627 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:13:16.874 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:17.170 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:16 smithi129 ceph-mon[30979]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T01:13:17.171 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:16 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T01:13:17.171 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:16 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:16 smithi194 ceph-mon[34941]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T01:13:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:16 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T01:13:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:16 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 -- 172.21.15.129:0/1999673451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290106f10 msgr2=0x7fa29010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 --2- 172.21.15.129:0/1999673451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290106f10 0x7fa29010ea90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa288009900 tx=0x7fa28802a410 comp rx=0 tx=0).stop 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 -- 172.21.15.129:0/1999673451 shutdown_connections 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 --2- 172.21.15.129:0/1999673451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290106f10 0x7fa29010ea90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 --2- 172.21.15.129:0/1999673451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290102790 0x7fa2901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 --2- 172.21.15.129:0/1999673451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290101de0 0x7fa2901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 -- 172.21.15.129:0/1999673451 >> 172.21.15.129:0/1999673451 conn(0x7fa2900fd8c0 msgr2=0x7fa2900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:17.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 -- 172.21.15.129:0/1999673451 shutdown_connections 2024-09-05T01:13:17.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.364+0000 7fa296dca640 1 -- 172.21.15.129:0/1999673451 wait complete. 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 Processor -- start 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 -- start start 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 0x7fa290198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290106f10 0x7fa29019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa290111e70 con 0x7fa290101de0 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa290111cf0 con 0x7fa290106f10 2024-09-05T01:13:17.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa296dca640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa290111ff0 con 0x7fa290102790 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa294b66640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa294b66640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43066/0 (socket says 172.21.15.129:43066) 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.365+0000 7fa294b66640 1 -- 172.21.15.129:0/3473398085 learned_addr learned my addr 172.21.15.129:0/3473398085 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa287fff640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 0x7fa290198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa295367640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290106f10 0x7fa29019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 -- 172.21.15.129:0/3473398085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 msgr2=0x7fa290198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 0x7fa290198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 -- 172.21.15.129:0/3473398085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290106f10 msgr2=0x7fa29019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:17.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290106f10 0x7fa29019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 -- 172.21.15.129:0/3473398085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29019d1d0 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa287fff640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 0x7fa290198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa294b66640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00cce0 tx=0x7fa28c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c00f650 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29019d4c0 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa28c00f7f0 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28c005280 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.366+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2901a4da0 con 0x7fa290101de0 2024-09-05T01:13:17.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.367+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2901021c0 con 0x7fa290101de0 2024-09-05T01:13:17.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.369+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa28c00fc30 con 0x7fa290101de0 2024-09-05T01:13:17.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.369+0000 7fa285ffb640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 0x7fa2780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:17.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.369+0000 7fa287fff640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 0x7fa2780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:17.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.369+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa28c0cf9a0 con 0x7fa290101de0 2024-09-05T01:13:17.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.370+0000 7fa287fff640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 0x7fa2780a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa280005e10 tx=0x7fa280005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:17.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.372+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa28c014030 con 0x7fa290101de0 2024-09-05T01:13:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:16 smithi183 ceph-mon[34589]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T01:13:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:16 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/928568922' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T01:13:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:16 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:17.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.573+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa290199580 con 0x7fa290101de0 2024-09-05T01:13:17.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.574+0000 7fa285ffb640 1 -- 172.21.15.129:0/3473398085 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11857 (secure 0 0 0) 0x7fa28c09c6c0 con 0x7fa290101de0 2024-09-05T01:13:17.575 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:17.575 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"btime":"2024-09-05T01:13:15:908403+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:15.572169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-05T01:13:17.575 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 17 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.576+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 msgr2=0x7fa2780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.576+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 0x7fa2780a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa280005e10 tx=0x7fa280005da0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.576+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 msgr2=0x7fa290198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.576+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00cce0 tx=0x7fa28c00a620 comp rx=0 tx=0).stop 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 shutdown_connections 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2780a2b80 0x7fa2780a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa290106f10 0x7fa29019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa290102790 0x7fa290198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 --2- 172.21.15.129:0/3473398085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa290101de0 0x7fa290198220 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:17.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 >> 172.21.15.129:0/3473398085 conn(0x7fa2900fd8c0 msgr2=0x7fa2900fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:17.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 shutdown_connections 2024-09-05T01:13:17.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:17.577+0000 7fa296dca640 1 -- 172.21.15.129:0/3473398085 wait complete. 2024-09-05T01:13:18.187 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3473398085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:18.187 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:17 smithi129 ceph-mon[30979]: pgmap v286: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:18.215 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:13:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3473398085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:17 smithi194 ceph-mon[34941]: pgmap v286: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3473398085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:17 smithi183 ceph-mon[34589]: pgmap v286: 129 pgs: 8 creating+peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:18.463 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:18.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 -- 172.21.15.129:0/3142589216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4108300 msgr2=0x7f0fe410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:18.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3142589216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4108300 0x7f0fe410ec70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009930 tx=0x7f0fdc02a420 comp rx=0 tx=0).stop 2024-09-05T01:13:18.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 -- 172.21.15.129:0/3142589216 shutdown_connections 2024-09-05T01:13:18.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3142589216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4108300 0x7f0fe410ec70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:18.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3142589216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4103780 0x7f0fe4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3142589216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 -- 172.21.15.129:0/3142589216 >> 172.21.15.129:0/3142589216 conn(0x7f0fe40fc7d0 msgr2=0x7f0fe40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 -- 172.21.15.129:0/3142589216 shutdown_connections 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.930+0000 7f0fe9348640 1 -- 172.21.15.129:0/3142589216 wait complete. 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 Processor -- start 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 -- start start 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe4100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4108300 0x7f0fe41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4112030 con 0x7f0fe4103780 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe4111eb0 con 0x7f0fe4102dd0 2024-09-05T01:13:18.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe9348640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe41121b0 con 0x7f0fe4108300 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe27fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43086/0 (socket says 172.21.15.129:43086) 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.931+0000 7f0fe27fc640 1 -- 172.21.15.129:0/3850609151 learned_addr learned my addr 172.21.15.129:0/3850609151 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe37fe640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4108300 0x7f0fe41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe4100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 -- 172.21.15.129:0/3850609151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4108300 msgr2=0x7f0fe41014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4108300 0x7f0fe41014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 -- 172.21.15.129:0/3850609151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 msgr2=0x7f0fe4100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe4100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 -- 172.21.15.129:0/3850609151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe41a6a90 con 0x7f0fe4103780 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:18.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe27fc640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00d750 tx=0x7f0fcc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc017070 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0fcc0054d0 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc004cf0 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe41a6d80 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.932+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe41a7210 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.933+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe41031b0 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.935+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0fcc004e90 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.936+0000 7f0fdbfff640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 0x7f0fd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.936+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 0x7f0fd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.936+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f0fcc0cefe0 con 0x7f0fe4103780 2024-09-05T01:13:18.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.936+0000 7f0fe2ffd640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 0x7f0fd00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009fa0 tx=0x7f0fd4009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:18.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:18.939+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fcc09bd00 con 0x7f0fe4103780 2024-09-05T01:13:19.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.103+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0fe410a290 con 0x7f0fe4103780 2024-09-05T01:13:19.105 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:18 smithi129 ceph-mon[30979]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T01:13:19.105 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:18 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:13:19.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.105+0000 7f0fdbfff640 1 -- 172.21.15.129:0/3850609151 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f0fcc09b600 con 0x7f0fe4103780 2024-09-05T01:13:19.106 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:19.107 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:13:19.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 msgr2=0x7f0fd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 0x7f0fd00a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009fa0 tx=0x7f0fd4009570 comp rx=0 tx=0).stop 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 msgr2=0x7f0fe4100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc00d750 tx=0x7f0fcc00dc10 comp rx=0 tx=0).stop 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 shutdown_connections 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0fd00a2b80 0x7f0fd00a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0fe4108300 0x7f0fe41014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0fe4103780 0x7f0fe4100f70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:19.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 --2- 172.21.15.129:0/3850609151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0fe4102dd0 0x7f0fe4100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:19.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 >> 172.21.15.129:0/3850609151 conn(0x7f0fe40fc7d0 msgr2=0x7f0fe4105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:19.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 shutdown_connections 2024-09-05T01:13:19.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:19.108+0000 7f0fe9348640 1 -- 172.21.15.129:0/3850609151 wait complete. 2024-09-05T01:13:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:18 smithi194 ceph-mon[34941]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T01:13:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:18 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:13:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:18 smithi183 ceph-mon[34589]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T01:13:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:18 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:13:19.805 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:13:20.060 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:20.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:19 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3850609151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:20.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:19 smithi129 ceph-mon[30979]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:19 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3850609151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:19 smithi194 ceph-mon[34941]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:20.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:19 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3850609151' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:19 smithi183 ceph-mon[34589]: pgmap v287: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:20.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- 172.21.15.129:0/3844843430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4100e20 msgr2=0x7f1fe4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:20.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/3844843430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4100e20 0x7f1fe4101200 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009900 tx=0x7f1fd402a480 comp rx=0 tx=0).stop 2024-09-05T01:13:20.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- 172.21.15.129:0/3844843430 shutdown_connections 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/3844843430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4102ef0 0x7f1fe410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/3844843430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102570 0x7f1fe41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/3844843430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4100e20 0x7f1fe4101200 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- 172.21.15.129:0/3844843430 >> 172.21.15.129:0/3844843430 conn(0x7f1fe40fc7d0 msgr2=0x7f1fe40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- 172.21.15.129:0/3844843430 shutdown_connections 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- 172.21.15.129:0/3844843430 wait complete. 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 Processor -- start 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.498+0000 7f1fe9d29640 1 -- start start 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4100e20 0x7f1fe41973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102ef0 0x7f1fe419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110e50 con 0x7f1fe4102570 2024-09-05T01:13:20.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110cd0 con 0x7f1fe4102ef0 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe9d29640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4110fd0 con 0x7f1fe4100e20 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43106/0 (socket says 172.21.15.129:43106) 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 -- 172.21.15.129:0/865324543 learned_addr learned my addr 172.21.15.129:0/865324543 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe3fff640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102ef0 0x7f1fe419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe37fe640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4100e20 0x7f1fe41973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 -- 172.21.15.129:0/865324543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4100e20 msgr2=0x7f1fe41973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4100e20 0x7f1fe41973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 -- 172.21.15.129:0/865324543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102ef0 msgr2=0x7f1fe419bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102ef0 0x7f1fe419bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 -- 172.21.15.129:0/865324543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe419c360 con 0x7f1fe4102570 2024-09-05T01:13:20.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fdbfff640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b4a0 tx=0x7f1fcc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.499+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc014070 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.500+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fcc010ac0 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.500+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe419c650 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.500+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017c30 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.500+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe41a3f30 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.501+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe4101200 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.502+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.502+0000 7f1fe17fa640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.502+0000 7f1fe37fe640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.502+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f1fcc0ce8c0 con 0x7f1fe4102570 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.503+0000 7f1fe37fe640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005ab0 tx=0x7f1fd4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:20.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.506+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fcc09b5e0 con 0x7f1fe4102570 2024-09-05T01:13:20.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.673+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fe4108b00 con 0x7f1fe4102570 2024-09-05T01:13:20.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.674+0000 7f1fe17fa640 1 -- 172.21.15.129:0/865324543 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f1fcc09aee0 con 0x7f1fe4102570 2024-09-05T01:13:20.675 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:20.675 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:13:20.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 msgr2=0x7f1fd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:20.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005ab0 tx=0x7f1fd4005a20 comp rx=0 tx=0).stop 2024-09-05T01:13:20.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 msgr2=0x7f1fe41978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:20.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00b4a0 tx=0x7f1fcc00b960 comp rx=0 tx=0).stop 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 shutdown_connections 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1fe4102ef0 0x7f1fe419bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1fe4102570 0x7f1fe41978f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 --2- 172.21.15.129:0/865324543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1fe4100e20 0x7f1fe41973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.676+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 >> 172.21.15.129:0/865324543 conn(0x7f1fe40fc7d0 msgr2=0x7f1fe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.677+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 shutdown_connections 2024-09-05T01:13:20.678 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:20.677+0000 7f1fe9d29640 1 -- 172.21.15.129:0/865324543 wait complete. 2024-09-05T01:13:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:20 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/865324543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:20 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/865324543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:21.370 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:13:21.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:20 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/865324543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:21.613 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:21.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:21 smithi129 ceph-mon[30979]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 -- 172.21.15.129:0/951335977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101df0 msgr2=0x7f6160105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 --2- 172.21.15.129:0/951335977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101df0 0x7f6160105920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f615c009900 tx=0x7f615c02a410 comp rx=0 tx=0).stop 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 -- 172.21.15.129:0/951335977 shutdown_connections 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 --2- 172.21.15.129:0/951335977 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616010dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 --2- 172.21.15.129:0/951335977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101df0 0x7f6160105920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 --2- 172.21.15.129:0/951335977 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101440 0x7f6160101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 -- 172.21.15.129:0/951335977 >> 172.21.15.129:0/951335977 conn(0x7f6160077210 msgr2=0x7f61600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 -- 172.21.15.129:0/951335977 shutdown_connections 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 -- 172.21.15.129:0/951335977 wait complete. 2024-09-05T01:13:22.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.109+0000 7f6167c44640 1 Processor -- start 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 -- start start 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 0x7f61601a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f616010f8b0 con 0x7f6160101440 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f616010f730 con 0x7f6160106050 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6167c44640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f616010fa30 con 0x7f6160101df0 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43124/0 (socket says 172.21.15.129:43124) 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 -- 172.21.15.129:0/4044097807 learned_addr learned my addr 172.21.15.129:0/4044097807 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61651df640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 0x7f61601a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:22.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61661e1640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 -- 172.21.15.129:0/4044097807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 msgr2=0x7f61601a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 0x7f61601a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 -- 172.21.15.129:0/4044097807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 msgr2=0x7f616019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 -- 172.21.15.129:0/4044097807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616019acd0 con 0x7f6160101440 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61651df640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 0x7f61601a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61661e1640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f61659e0640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f615400d950 tx=0x7f615400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.110+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6154017070 con 0x7f6160101440 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.111+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616019afc0 con 0x7f6160101440 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.111+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6154009c30 con 0x7f6160101440 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.111+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615400f620 con 0x7f6160101440 2024-09-05T01:13:22.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.111+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61601a72b0 con 0x7f6160101440 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.112+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6160101820 con 0x7f6160101440 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.113+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f61540109a0 con 0x7f6160101440 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.113+0000 7f6152ffd640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 0x7f614c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.113+0000 7f61651df640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 0x7f614c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.113+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f61540cef80 con 0x7f6160101440 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.114+0000 7f61651df640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 0x7f614c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f615c031440 tx=0x7f615c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:22.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.117+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6154009de0 con 0x7f6160101440 2024-09-05T01:13:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:21 smithi194 ceph-mon[34941]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:22.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.316+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61601a7480 con 0x7f6160101440 2024-09-05T01:13:22.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.317+0000 7f6152ffd640 1 -- 172.21.15.129:0/4044097807 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11857 (secure 0 0 0) 0x7f615409bca0 con 0x7f6160101440 2024-09-05T01:13:22.318 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:22.318 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"btime":"2024-09-05T01:13:15:908403+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24433,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24439,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:15.572169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24488,"qdb_cluster":[24488]},"id":1}]} 2024-09-05T01:13:22.318 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 17 2024-09-05T01:13:22.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 msgr2=0x7f614c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 0x7f614c0a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f615c031440 tx=0x7f615c005a20 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 msgr2=0x7f61601a03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f615400d950 tx=0x7f615400de10 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 shutdown_connections 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f614c0a2b80 0x7f614c0a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6160106050 0x7f616019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6160101df0 0x7f61601a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 --2- 172.21.15.129:0/4044097807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6160101440 0x7f61601a03e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.319+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 >> 172.21.15.129:0/4044097807 conn(0x7f6160077210 msgr2=0x7f61601039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.320+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 shutdown_connections 2024-09-05T01:13:22.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:22.320+0000 7f6167c44640 1 -- 172.21.15.129:0/4044097807 wait complete. 2024-09-05T01:13:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:21 smithi183 ceph-mon[34589]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:22.952 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:22 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4044097807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:22.982 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:13:23.230 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:22 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4044097807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:22 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4044097807' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 -- 172.21.15.129:0/345855896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 msgr2=0x7f217810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 --2- 172.21.15.129:0/345855896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f217810d950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2170009cd0 tx=0x7f217002a8e0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 -- 172.21.15.129:0/345855896 shutdown_connections 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 --2- 172.21.15.129:0/345855896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f217810d950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 --2- 172.21.15.129:0/345855896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 --2- 172.21.15.129:0/345855896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 0x7f2178106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 -- 172.21.15.129:0/345855896 >> 172.21.15.129:0/345855896 conn(0x7f21780fc7a0 msgr2=0x7f21780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 -- 172.21.15.129:0/345855896 shutdown_connections 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.683+0000 7f217ce2e640 1 -- 172.21.15.129:0/345855896 wait complete. 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 Processor -- start 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 -- start start 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:23.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 0x7f217819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2178110d30 con 0x7f2178101420 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2178110bb0 con 0x7f2178100a80 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f217ce2e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2178110eb0 con 0x7f2178106a80 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f216ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f216ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59122/0 (socket says 172.21.15.129:59122) 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.684+0000 7f216ffff640 1 -- 172.21.15.129:0/4143964858 learned_addr learned my addr 172.21.15.129:0/4143964858 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f2176d9d640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 0x7f217819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f217659c640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 -- 172.21.15.129:0/4143964858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 msgr2=0x7f217819bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 0x7f217819bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 -- 172.21.15.129:0/4143964858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 msgr2=0x7f2178197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 -- 172.21.15.129:0/4143964858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217819c1e0 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f217659c640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216ffff640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2160009970 tx=0x7f2160009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000d020 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217819c4d0 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f216000d1c0 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000cc50 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.685+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217806a790 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.686+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2178100ee0 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.688+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f216000c460 con 0x7f2178101420 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.688+0000 7f216f7fe640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 0x7f21640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:23.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.688+0000 7f217659c640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 0x7f21640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:23.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.688+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f21600d0be0 con 0x7f2178101420 2024-09-05T01:13:23.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.689+0000 7f217659c640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 0x7f21640a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2168009ad0 tx=0x7f2168005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:23.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.692+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f216009d900 con 0x7f2178101420 2024-09-05T01:13:23.861 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.859+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f217806b1e0 con 0x7f2178101420 2024-09-05T01:13:23.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.860+0000 7f216f7fe640 1 -- 172.21.15.129:0/4143964858 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f216009d200 con 0x7f2178101420 2024-09-05T01:13:23.862 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:23.862 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:13:23.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 msgr2=0x7f21640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:23.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 0x7f21640a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2168009ad0 tx=0x7f2168005f90 comp rx=0 tx=0).stop 2024-09-05T01:13:23.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 msgr2=0x7f2178197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:23.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2160009970 tx=0x7f2160009e30 comp rx=0 tx=0).stop 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 shutdown_connections 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21640a2b80 0x7f21640a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2178106a80 0x7f217819bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2178101420 0x7f2178197770 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 --2- 172.21.15.129:0/4143964858 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2178100a80 0x7f2178197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.862+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 >> 172.21.15.129:0/4143964858 conn(0x7f21780fc7a0 msgr2=0x7f21780fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.863+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 shutdown_connections 2024-09-05T01:13:23.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:23.863+0000 7f217ce2e640 1 -- 172.21.15.129:0/4143964858 wait complete. 2024-09-05T01:13:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:23 smithi129 ceph-mon[30979]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:23 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4143964858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:23 smithi194 ceph-mon[34941]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:23 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4143964858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:23 smithi183 ceph-mon[34589]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:23 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4143964858' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:24.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-05T01:13:06.500509+0000', 'modified': '2024-09-05T01:13:15.572169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/3533858614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3533858614}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3533858614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24488, 'qdb_cluster': [24488]} 2024-09-05T01:13:24.625 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T01:13:24.626 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T01:13:24.871 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:25.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 -- 172.21.15.129:0/903277617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc102570 msgr2=0x7f30cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 --2- 172.21.15.129:0/903277617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc102570 0x7f30cc1029b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009900 tx=0x7f30c002a480 comp rx=0 tx=0).stop 2024-09-05T01:13:25.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 -- 172.21.15.129:0/903277617 shutdown_connections 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 --2- 172.21.15.129:0/903277617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 0x7f30cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 --2- 172.21.15.129:0/903277617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc102570 0x7f30cc1029b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 --2- 172.21.15.129:0/903277617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc100e20 0x7f30cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 -- 172.21.15.129:0/903277617 >> 172.21.15.129:0/903277617 conn(0x7f30cc0fc7d0 msgr2=0x7f30cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 -- 172.21.15.129:0/903277617 shutdown_connections 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.352+0000 7f30d11ac640 1 -- 172.21.15.129:0/903277617 wait complete. 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 Processor -- start 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 -- start start 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 0x7f30cc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 0x7f30cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc110ed0 con 0x7f30cc100e20 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc110d50 con 0x7f30cc102570 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.353+0000 7f30d11ac640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc111050 con 0x7f30cc102ef0 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59130/0 (socket says 172.21.15.129:59130) 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 -- 172.21.15.129:0/2724713512 learned_addr learned my addr 172.21.15.129:0/2724713512 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30ca59c640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 0x7f30cc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 0x7f30cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 -- 172.21.15.129:0/2724713512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 msgr2=0x7f30cc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 0x7f30cc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 -- 172.21.15.129:0/2724713512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 msgr2=0x7f30cc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 0x7f30cc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 -- 172.21.15.129:0/2724713512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc19d480 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30ca59c640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 0x7f30cc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30cad9d640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00ba50 tx=0x7f30bc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30b3fff640 1 -- 172.21.15.129:0/2724713512 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc019070 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc19d770 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30b3fff640 1 -- 172.21.15.129:0/2724713512 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30bc00cbe0 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30b3fff640 1 -- 172.21.15.129:0/2724713512 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0075d0 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.354+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a5050 con 0x7f30cc100e20 2024-09-05T01:13:25.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.357+0000 7f30b3fff640 1 -- 172.21.15.129:0/2724713512 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f30bc01c070 con 0x7f30cc100e20 2024-09-05T01:13:25.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.357+0000 7f30d3585740 1 Processor -- start 2024-09-05T01:13:25.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.357+0000 7f30b3fff640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 0x7f30b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.357+0000 7f30d3585740 1 -- start start 2024-09-05T01:13:25.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 0x7f30b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa79d0 0x5574bbfa7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30b3fff640 1 -- 172.21.15.129:0/2724713512 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f30bc0ce950 con 0x7f30cc100e20 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfb4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574bbf0e090 con 0x5574bbfb2bb0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574bbeb5a70 con 0x5574bbfa82f0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30d3585740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574bbf0a050 con 0x5574bbfa79d0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30ca59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfb4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30ca59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56414/0 (socket says 172.21.15.129:56414) 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30ca59c640 1 -- 172.21.15.129:0/2364677321 learned_addr learned my addr 172.21.15.129:0/2364677321 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cad9d640 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa79d0 0x5574bbfa7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 0x7f30b40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b880 tx=0x7f30c400bc80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 -- 172.21.15.129:0/2364677321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa79d0 msgr2=0x5574bbfa7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa79d0 0x5574bbfa7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 -- 172.21.15.129:0/2364677321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 msgr2=0x5574bbfb2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 -- 172.21.15.129:0/2364677321 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bbec2130 con 0x5574bbfb2bb0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30ca59c640 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.358+0000 7f30cb59e640 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfb4fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30c400d920 tx=0x7f30c4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9844afcf29d46837 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30b17fa640 1 -- 172.21.15.129:0/2364677321 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c401a040 con 0x5574bbfb2bb0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30b17fa640 1 -- 172.21.15.129:0/2364677321 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c4023070 con 0x5574bbfb2bb0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30b17fa640 1 -- 172.21.15.129:0/2364677321 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c401ebb0 con 0x5574bbfb2bb0 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 -- 172.21.15.129:0/2364677321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 msgr2=0x5574bbfb4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfb4fa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f30c400d920 tx=0x7f30c4018000 comp rx=0 tx=0).stop 2024-09-05T01:13:25.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 -- 172.21.15.129:0/2364677321 shutdown_connections 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfb4fa0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfb2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 --2- 172.21.15.129:0/2364677321 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa79d0 0x5574bbfa7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.359+0000 7f30d3585740 1 -- 172.21.15.129:0/2364677321 >> 172.21.15.129:0/2364677321 conn(0x5574bbfa3f60 msgr2=0x5574bbfa4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- 172.21.15.129:0/2364677321 shutdown_connections 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- 172.21.15.129:0/2364677321 wait complete. 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- start start 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 Processor -- start 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfaf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfb2bb0 0x5574bbfafca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfb9b90 con 0x5574bbfa79d0 2024-09-05T01:13:25.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574bbec8f20 con 0x5574bbfa82f0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfb9d10 con 0x5574bbfb2bb0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30d3585740 10 client.0 _ll_register_callbacks cb 0x5574bbe92030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59146/0 (socket says 172.21.15.129:59146) 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.360+0000 7f30cad9d640 1 -- 172.21.15.129:0/1479338620 learned_addr learned my addr 172.21.15.129:0/1479338620 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 -- 172.21.15.129:0/1479338620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfb2bb0 msgr2=0x5574bbfafca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfb2bb0 0x5574bbfafca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30ca59c640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfaf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 -- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 msgr2=0x5574bbfaf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfaf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 -- 172.21.15.129:0/1479338620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bbc7f2b0 con 0x5574bbfa79d0 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30ca59c640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfaf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:25.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30cad9d640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009fd0 tx=0x7f30bc09afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8f1cbb5113042b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0a36f0 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f30bc0a3890 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c4023070 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f30bc0b9b50 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30d3585740 10 client.14724 fetch_fsmap learned FSMap version 17 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5574bbfc54b0 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c4023070 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.361+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f30bc099dd0 con 0x5574bbfa79d0 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 10 client.14724 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 10 client.14724 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:13:25.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 20 client.14724 populate_metadata read hostname 'smithi129' 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bbfc8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 4 client.14724 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574bbf1b600 con 0x5574bbfc7b10 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30ca59c640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bbfc8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30ca59c640 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bbfc8f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1429022acff6f7c3 server_cookie=dabcf9196e1b7ebb in_seq=0 out_seq=0 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.362+0000 7f30aeff5640 10 client.14724 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.363+0000 7f30aeff5640 1 -- 172.21.15.129:0/1479338620 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574bbf1b600 con 0x5574bbfc7b10 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.363+0000 7f30aeff5640 10 client.14724 handle_command_reply: tid=0 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.363+0000 7f30d3585740 1 client.14724 shutdown 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.363+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 msgr2=0x5574bbfb23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.363+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009fd0 tx=0x7f30bc09afd0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 shutdown_connections 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bbfc8f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfb2bb0 0x5574bbfafca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30aeff5640 0 client.14724 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa82f0 0x5574bbfaf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 --2- 172.21.15.129:0/1479338620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfb23c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 >> 172.21.15.129:0/1479338620 conn(0x5574bbfa3f60 msgr2=0x5574bbfa9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 shutdown_connections 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 1 -- 172.21.15.129:0/1479338620 wait complete. 2024-09-05T01:13:25.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.364+0000 7f30d3585740 20 client.14724 trim_cache size 0 max 16384 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 Processor -- start 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 -- start start 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 0x5574bbfaf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc72c0 con 0x5574bbfb2bb0 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc7990 con 0x5574bbfa79d0 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30d3585740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc8060 con 0x5574bbfa82f0 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30cb59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30cb59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59154/0 (socket says 172.21.15.129:59154) 2024-09-05T01:13:25.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.381+0000 7f30cb59e640 1 -- 172.21.15.129:0/3914066443 learned_addr learned my addr 172.21.15.129:0/3914066443 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cad9d640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 0x5574bbfaf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30ca59c640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 -- 172.21.15.129:0/3914066443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 msgr2=0x5574bbfd1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfd1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 -- 172.21.15.129:0/3914066443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 msgr2=0x5574bbfaf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 0x5574bbfaf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 -- 172.21.15.129:0/3914066443 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574bbfb02a0 con 0x5574bbfb2bb0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cad9d640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 0x5574bbfaf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30cb59e640 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b2d0 tx=0x7f30c400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ac38c358d78eab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30b0ff9640 1 -- 172.21.15.129:0/3914066443 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c401a040 con 0x5574bbfb2bb0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30b0ff9640 1 -- 172.21.15.129:0/3914066443 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c402c070 con 0x5574bbfb2bb0 2024-09-05T01:13:25.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.382+0000 7f30b0ff9640 1 -- 172.21.15.129:0/3914066443 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c401e670 con 0x5574bbfb2bb0 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- 172.21.15.129:0/3914066443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 msgr2=0x5574bbfd17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b2d0 tx=0x7f30c400bef0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- 172.21.15.129:0/3914066443 shutdown_connections 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfb2bb0 0x5574bbfd17a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- 172.21.15.129:0/3914066443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfa79d0 0x5574bbfaf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- 172.21.15.129:0/3914066443 >> 172.21.15.129:0/3914066443 conn(0x5574bbfa3f60 msgr2=0x5574bbfaf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- 172.21.15.129:0/3914066443 shutdown_connections 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- 172.21.15.129:0/3914066443 wait complete. 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- start start 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 Processor -- start 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfcac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfb2bb0 0x5574bc030210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc8e00 con 0x5574bbfa79d0 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc8730 con 0x5574bbfb2bb0 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5574bbfc94d0 con 0x5574bbfa82f0 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30d3585740 10 client.0 _ll_register_callbacks cb 0x7f30bc0bb0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59170/0 (socket says 172.21.15.129:59170) 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.383+0000 7f30cad9d640 1 -- 172.21.15.129:0/953749281 learned_addr learned my addr 172.21.15.129:0/953749281 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30ca59c640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfcac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cb59e640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfb2bb0 0x5574bc030210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 -- 172.21.15.129:0/953749281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 msgr2=0x5574bbfcac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfcac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 -- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfb2bb0 msgr2=0x5574bc030210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfb2bb0 0x5574bc030210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 -- 172.21.15.129:0/953749281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574bc030750 con 0x5574bbfa79d0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30cad9d640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0bb180 tx=0x7f30bc009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1664a02f770ef4da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc099dd0 con 0x5574bbfa79d0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f30bc0a05e0 con 0x5574bbfa79d0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c402c070 con 0x5574bbfa79d0 2024-09-05T01:13:25.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f30bc099f70 con 0x5574bbfa79d0 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30d3585740 10 client.14730 fetch_fsmap learned FSMap version 17 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5574bc031970 con 0x5574bbfa79d0 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30c402c070 con 0x5574bbfa79d0 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.384+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f30bc096c60 con 0x5574bbfa79d0 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 10 client.14730 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 10 client.14730 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 20 client.14730 populate_metadata read hostname 'smithi129' 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bc034ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 4 client.14730 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "status"}) -- 0x5574bbf15b10 con 0x5574bbfc7b10 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30ca59c640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bc034ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:25.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30aeff5640 10 client.14730 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.385+0000 7f30ca59c640 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bc034ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f51c9d327ed5c7b server_cookie=ba4c53f2e908ece in_seq=0 out_seq=0 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30aeff5640 1 -- 172.21.15.129:0/953749281 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5574bbf15b10 con 0x5574bbfc7b10 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30aeff5640 10 client.14730 handle_command_reply: tid=0 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 client.14730 shutdown 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 msgr2=0x5574bbfca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0bb180 tx=0x7f30bc009930 comp rx=0 tx=0).stop 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 shutdown_connections 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5574bbfc7b10 0x5574bc034ff0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30aeff5640 0 client.14730 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5574bbfb2bb0 0x5574bc030210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5574bbfa82f0 0x5574bbfcac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 --2- 172.21.15.129:0/953749281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574bbfa79d0 0x5574bbfca700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.393 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 >> 172.21.15.129:0/953749281 conn(0x5574bbfa3f60 msgr2=0x5574bbfaa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 shutdown_connections 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 1 -- 172.21.15.129:0/953749281 wait complete. 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d3585740 20 client.14730 trim_cache size 0 max 16384 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24488, 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:13:25.394 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 18.500792405999999, 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 17, 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 86, 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 29.353865763999998 2024-09-05T01:13:25.395 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.386+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 msgr2=0x7f30b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 0x7f30b40a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b880 tx=0x7f30c400bc80 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 msgr2=0x7f30cc198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00ba50 tx=0x7f30bc00bf10 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 shutdown_connections 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f30b40a2b80 0x7f30b40a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f30cc102ef0 0x7f30cc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30cc102570 0x7f30cc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 --2- 172.21.15.129:0/2724713512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f30cc100e20 0x7f30cc198570 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 >> 172.21.15.129:0/2724713512 conn(0x7f30cc0fc7d0 msgr2=0x7f30cc069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 shutdown_connections 2024-09-05T01:13:25.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:25.387+0000 7f30d11ac640 1 -- 172.21.15.129:0/2724713512 wait complete. 2024-09-05T01:13:26.192 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T01:13:26.193 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:26.193 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T01:13:26.437 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:26.860 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:26 smithi129 ceph-mon[30979]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:26.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 -- 172.21.15.129:0/2887163760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174106f10 msgr2=0x7f217410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:26.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 --2- 172.21.15.129:0/2887163760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174106f10 0x7f217410ea90 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f216c009ca0 tx=0x7f216c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 -- 172.21.15.129:0/2887163760 shutdown_connections 2024-09-05T01:13:26.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 --2- 172.21.15.129:0/2887163760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174106f10 0x7f217410ea90 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 --2- 172.21.15.129:0/2887163760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174102790 0x7f21741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 --2- 172.21.15.129:0/2887163760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174101de0 0x7f21741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 -- 172.21.15.129:0/2887163760 >> 172.21.15.129:0/2887163760 conn(0x7f21740fd880 msgr2=0x7f21740ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:26.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 -- 172.21.15.129:0/2887163760 shutdown_connections 2024-09-05T01:13:26.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.862+0000 7f217a05f640 1 -- 172.21.15.129:0/2887163760 wait complete. 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 Processor -- start 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 -- start start 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 0x7f21740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 0x7f2174074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2174111e20 con 0x7f2174102790 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2174111ca0 con 0x7f2174101de0 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f217a05f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2174111fa0 con 0x7f2174106f10 2024-09-05T01:13:26.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f2172ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f2172ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59190/0 (socket says 172.21.15.129:59190) 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.863+0000 7f2172ffd640 1 -- 172.21.15.129:0/1654730110 learned_addr learned my addr 172.21.15.129:0/1654730110 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2173fff640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 0x7f2174074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f21737fe640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 0x7f21740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 -- 172.21.15.129:0/1654730110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 msgr2=0x7f2174074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 0x7f2174074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 -- 172.21.15.129:0/1654730110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 msgr2=0x7f21740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 0x7f21740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:26.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 -- 172.21.15.129:0/1654730110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21741a6740 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f21737fe640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 0x7f21740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2173fff640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 0x7f2174074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2172ffd640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f216800c920 tx=0x7f216800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168007c70 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2168007e10 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2168018820 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21741a6a30 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.864+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21741a6fc0 con 0x7f2174102790 2024-09-05T01:13:26.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.865+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21741021c0 con 0x7f2174102790 2024-09-05T01:13:26.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.866+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f21680189c0 con 0x7f2174102790 2024-09-05T01:13:26.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.867+0000 7f2170ff9640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 0x7f215c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:26.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.867+0000 7f21737fe640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 0x7f215c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:26.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.867+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f21680cf3b0 con 0x7f2174102790 2024-09-05T01:13:26.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.867+0000 7f21737fe640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 0x7f215c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2164009b40 tx=0x7f2164005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:26.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:26.871+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2168014030 con 0x7f2174102790 2024-09-05T01:13:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.073+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f217402d7c0 con 0x7f2174102790 2024-09-05T01:13:27.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.074+0000 7f2170ff9640 1 -- 172.21.15.129:0/1654730110 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f216800de20 con 0x7f2174102790 2024-09-05T01:13:27.075 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T01:13:27.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 msgr2=0x7f215c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:27.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 0x7f215c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2164009b40 tx=0x7f2164005e50 comp rx=0 tx=0).stop 2024-09-05T01:13:27.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 msgr2=0x7f2174074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:27.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f216800c920 tx=0x7f216800cde0 comp rx=0 tx=0).stop 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 shutdown_connections 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f215c0a2c00 0x7f215c0a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2174106f10 0x7f2174074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2174102790 0x7f2174074a30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 --2- 172.21.15.129:0/1654730110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2174101de0 0x7f21740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 >> 172.21.15.129:0/1654730110 conn(0x7f21740fd880 msgr2=0x7f21740feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 shutdown_connections 2024-09-05T01:13:27.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27.076+0000 7f217a05f640 1 -- 172.21.15.129:0/1654730110 wait complete. 2024-09-05T01:13:27.113 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:26 smithi183 ceph-mon[34589]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:26 smithi194 ceph-mon[34941]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:27.833 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:27 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1654730110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:13:27.861 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-05T01:13:27.861 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T01:13:27.862 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T01:13:27.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:27.862 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:13:27.879 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:13:27.880 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T01:13:27.881 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-09-05T01:13:27.881 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link noprefixroute 2024-09-05T01:13:27.881 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:13:27.882 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T01:13:27.882 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:13:27.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:13:28.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:28.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:28.086 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T01:13:28.112 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-05T01:13:28.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:28.113 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-09-05T01:13:28.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:27 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1654730110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:13:28.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:27 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1654730110' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:13:28.195 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-09-05T01:13:28.195 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T01:13:28.195 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T01:13:28.195 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T01:13:28.195 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T01:13:28.196 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:13:28.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:13:28.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:28.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:28.464 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-05T01:13:28.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:28.481 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-09-05T01:13:28.536 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:13:28.536 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:13:28.536 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:13:28.536 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-05T01:13:28.537 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:13:28.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:13:28.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:28.654 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-05T01:13:28.654 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:13:28.654 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-05T01:13:28.655 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:13:28.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:13:28.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:28.791 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:13:28.791 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:13:28.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-09-05T01:13:28.792 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T01:13:28.792 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:13:28.839 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:13:28.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:28.869 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-05T01:13:28.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T01:13:28.870 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:28.898 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T01:13:28.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T01:13:28.898 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:28.954 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-09-05T01:13:28.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:28 smithi129 ceph-mon[30979]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:29.016 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T01:13:29.072 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-09-05T01:13:29.073 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-09-05T01:13:29.074 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/bc5b16d8b1bb7d929277aeb93eb862115f1162cf050ea5a605472c71523282d1/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a9d8e443514e1b43451c70231c10b54477b248cfb48ee2100dd5954906931016/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/0 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/9e3a8c301999c6a31f1cb49349caa5c281e688a2ff7873f3bc02f5e045ae807c/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a27ef43250c18d4f0de027197eea562db771db73ec4dd5a76ede93f1de32407f/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/76b7e3cc342992e8d2f38e13eeb34d9084309dfefcef60ecbc2ab34f036ac6cf/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/abc3d332c7e32f435eccf9439fd3dc72aaaf6ccf18adab417caf7f5b8de7ddea/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/38493fa094ff6b25db46252e4ada6ebc9ffe48f85e05e75fd9ef24d93242d781/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/e07dcd732625a75fd5ea12a2f6c01c27646a9e50d72e8b1a488036c5490642f7/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/b0948983ff36c2ab972d19716e21d96c48dd67a358dfd822f93154fae5f612b4/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/81f58d290eca11f257b1d7dc4c0be37b6df06a645878a08ac32e720ae5d141b6/merged 2024-09-05T01:13:29.075 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-09-05T01:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:13:29.076 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:13:29.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:29.076 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-09-05T01:13:29.128 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T01:13:29.128 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-05T01:13:29.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:28 smithi183 ceph-mon[34589]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:29.174 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-09-05T01:13:29.199 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T01:13:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:28 smithi194 ceph-mon[34941]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:29.255 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-09-05T01:13:29.255 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-09-05T01:13:29.255 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-09-05T01:13:29.255 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-09-05T01:13:29.256 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-09-05T01:13:29.257 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/bc5b16d8b1bb7d929277aeb93eb862115f1162cf050ea5a605472c71523282d1/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a9d8e443514e1b43451c70231c10b54477b248cfb48ee2100dd5954906931016/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/0 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/9e3a8c301999c6a31f1cb49349caa5c281e688a2ff7873f3bc02f5e045ae807c/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a27ef43250c18d4f0de027197eea562db771db73ec4dd5a76ede93f1de32407f/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/76b7e3cc342992e8d2f38e13eeb34d9084309dfefcef60ecbc2ab34f036ac6cf/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/abc3d332c7e32f435eccf9439fd3dc72aaaf6ccf18adab417caf7f5b8de7ddea/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/38493fa094ff6b25db46252e4ada6ebc9ffe48f85e05e75fd9ef24d93242d781/merged 2024-09-05T01:13:29.258 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/e07dcd732625a75fd5ea12a2f6c01c27646a9e50d72e8b1a488036c5490642f7/merged 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/b0948983ff36c2ab972d19716e21d96c48dd67a358dfd822f93154fae5f612b4/merged 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/81f58d290eca11f257b1d7dc4c0be37b6df06a645878a08ac32e720ae5d141b6/merged 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:13:29.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:29.259 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-09-05T01:13:29.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:2024-09-05T01:13:29.275+0000 7f418a53c480 -1 init, newargv = 0x557df82d2d00 newargc=15 2024-09-05T01:13:29.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[74775]: starting ceph client 2024-09-05T01:13:29.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[74775]: starting fuse 2024-09-05T01:13:29.292 INFO:teuthology.orchestra.run.smithi129.stdout:121 2024-09-05T01:13:29.292 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-09-05T01:13:29.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-09-05T01:13:29.292 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-09-05T01:13:29.292 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-09-05T01:13:29.292 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-05T01:13:29.293 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T01:13:29.294 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.admin")) 2024-09-05T01:13:29.295 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-09-05T01:13:29.410 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.admin.74775.asok 2024-09-05T01:13:29.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:29.421 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74775.asok 2024-09-05T01:13:29.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:29.421 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74775.asok status 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "admin", 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-05T01:13:29.555 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "74775", 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/" 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 14742, 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 14742 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 751179244 2024-09-05T01:13:29.556 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 751179244 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.14742 192.168.0.1:0/751179244", 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.1:0/751179244", 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 1, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 17, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 86, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-09-05T01:13:29.557 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:13:29.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:29.564 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-09-05T01:13:29.564 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:29.580 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-09-05T01:13:29.580 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:29.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:29.580 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:29.646 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-05T01:13:29.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T01:13:29.753 INFO:teuthology.orchestra.run.smithi129.stdout:# file: . 2024-09-05T01:13:29.753 INFO:teuthology.orchestra.run.smithi129.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-05T01:13:29.753 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:29.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:13:29.759 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:13:29.759 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T01:13:29.776 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T01:13:30.072 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:30.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- 172.21.15.129:0/408375284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204106a80 msgr2=0x7f8204106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:30.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 --2- 172.21.15.129:0/408375284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204106a80 0x7f8204106e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8200009930 tx=0x7f820002a420 comp rx=0 tx=0).stop 2024-09-05T01:13:30.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- 172.21.15.129:0/408375284 shutdown_connections 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 --2- 172.21.15.129:0/408375284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204101420 0x7f820410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 --2- 172.21.15.129:0/408375284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204100a80 0x7f8204100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 --2- 172.21.15.129:0/408375284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204106a80 0x7f8204106e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- 172.21.15.129:0/408375284 >> 172.21.15.129:0/408375284 conn(0x7f82040fc7a0 msgr2=0x7f82040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- 172.21.15.129:0/408375284 shutdown_connections 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- 172.21.15.129:0/408375284 wait complete. 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 Processor -- start 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.546+0000 7f820bc07640 1 -- start start 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204101420 0x7f8204199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204106a80 0x7f82041a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8204110d30 con 0x7f8204100a80 2024-09-05T01:13:30.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8204110bb0 con 0x7f8204101420 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820bc07640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8204110eb0 con 0x7f8204106a80 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59212/0 (socket says 172.21.15.129:59212) 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 -- 172.21.15.129:0/998222747 learned_addr learned my addr 172.21.15.129:0/998222747 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f820a1a4640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204106a80 0x7f82041a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82091a2640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204101420 0x7f8204199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 -- 172.21.15.129:0/998222747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204106a80 msgr2=0x7f82041a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204106a80 0x7f82041a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 -- 172.21.15.129:0/998222747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204101420 msgr2=0x7f8204199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204101420 0x7f8204199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 -- 172.21.15.129:0/998222747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82041a47a0 con 0x7f8204100a80 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.547+0000 7f82099a3640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8200004770 tx=0x7f82000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:30.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.548+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820003f070 con 0x7f8204100a80 2024-09-05T01:13:30.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.548+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f820003b460 con 0x7f8204100a80 2024-09-05T01:13:30.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.548+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82041a4970 con 0x7f8204100a80 2024-09-05T01:13:30.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.548+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8200035070 con 0x7f8204100a80 2024-09-05T01:13:30.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.548+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82041a4e20 con 0x7f8204100a80 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.549+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8204100ee0 con 0x7f8204100a80 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.550+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f820003b610 con 0x7f8204100a80 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.550+0000 7f81faffd640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.551+0000 7f82091a2640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.551+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f82000ef590 con 0x7f8204100a80 2024-09-05T01:13:30.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.551+0000 7f82091a2640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 0x7f81f00a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f81f4005fd0 tx=0x7f81f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:30.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.555+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82000bc2b0 con 0x7f8204100a80 2024-09-05T01:13:30.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.734+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8204108a60 con 0x7f8204100a80 2024-09-05T01:13:30.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.734+0000 7f81faffd640 1 -- 172.21.15.129:0/998222747 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+601 (secure 0 0 0) 0x7f82000bbbb0 con 0x7f8204100a80 2024-09-05T01:13:30.736 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:30.736 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:30.736 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/665628602 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3304384691 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/2755277182 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4057892175 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4214690776 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3403026756 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:30.737 INFO:teuthology.orchestra.run.smithi129.stderr:listed 10 entries 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 msgr2=0x7f81f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 0x7f81f00a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f81f4005fd0 tx=0x7f81f4005850 comp rx=0 tx=0).stop 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 msgr2=0x7f82041991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8200004770 tx=0x7f82000042f0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 shutdown_connections 2024-09-05T01:13:30.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f81f00a2c00 0x7f81f00a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8204106a80 0x7f82041a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8204101420 0x7f8204199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 --2- 172.21.15.129:0/998222747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8204100a80 0x7f82041991e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 >> 172.21.15.129:0/998222747 conn(0x7f82040fc7a0 msgr2=0x7f82040fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 shutdown_connections 2024-09-05T01:13:30.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:30.737+0000 7f820bc07640 1 -- 172.21.15.129:0/998222747 wait complete. 2024-09-05T01:13:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:30 smithi129 ceph-mon[30979]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:30.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/998222747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:30 smithi183 ceph-mon[34589]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/998222747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:30 smithi194 ceph-mon[34941]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/998222747' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:31.469 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:13:31.469 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T01:13:31.488 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T01:13:31.778 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:32.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 -- 172.21.15.129:0/2611637430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4106a80 msgr2=0x7f13a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/2611637430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4106a80 0x7f13a4106e60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009930 tx=0x7f13a002a420 comp rx=0 tx=0).stop 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 -- 172.21.15.129:0/2611637430 shutdown_connections 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/2611637430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4101420 0x7f13a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/2611637430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4100a80 0x7f13a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/2611637430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4106a80 0x7f13a4106e60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 -- 172.21.15.129:0/2611637430 >> 172.21.15.129:0/2611637430 conn(0x7f13a40fc7a0 msgr2=0x7f13a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 -- 172.21.15.129:0/2611637430 shutdown_connections 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.220+0000 7f13ac1e4640 1 -- 172.21.15.129:0/2611637430 wait complete. 2024-09-05T01:13:32.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 Processor -- start 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 -- start start 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4100a80 0x7f13a4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4106a80 0x7f13a419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4110d30 con 0x7f13a4101420 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4110bb0 con 0x7f13a4100a80 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13ac1e4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4110eb0 con 0x7f13a4106a80 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13a977f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13a977f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50252/0 (socket says 172.21.15.129:50252) 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13a977f640 1 -- 172.21.15.129:0/3044441834 learned_addr learned my addr 172.21.15.129:0/3044441834 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13aa781640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4106a80 0x7f13a419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:32.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.221+0000 7f13a9f80640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4100a80 0x7f13a4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 -- 172.21.15.129:0/3044441834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4106a80 msgr2=0x7f13a419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4106a80 0x7f13a419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 -- 172.21.15.129:0/3044441834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4100a80 msgr2=0x7f13a4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4100a80 0x7f13a4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 -- 172.21.15.129:0/3044441834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a419d2d0 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13a977f640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f139800c970 tx=0x7f139800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398007bb0 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a419d5c0 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1398007d50 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139800f450 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.222+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41a4ea0 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.223+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a4100ee0 con 0x7f13a4101420 2024-09-05T01:13:32.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.224+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1398004f20 con 0x7f13a4101420 2024-09-05T01:13:32.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.225+0000 7f1396ffd640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 0x7f13900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:32.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.225+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f13980cf000 con 0x7f13a4101420 2024-09-05T01:13:32.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.225+0000 7f13a9f80640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 0x7f13900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:32.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.225+0000 7f13a9f80640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 0x7f13900a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13a0000c00 tx=0x7f13a0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:32.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.229+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1398014030 con 0x7f13a4101420 2024-09-05T01:13:32.410 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.409+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f13a41a5140 con 0x7f13a4101420 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.409+0000 7f1396ffd640 1 -- 172.21.15.129:0/3044441834 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+601 (secure 0 0 0) 0x7f139809bd20 con 0x7f13a4101420 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/665628602 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3304384691 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/2755277182 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4057892175 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:32.411 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4214690776 2024-09-06T01:06:17.664286+0000 2024-09-05T01:13:32.412 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:32.412 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3403026756 2024-09-06T01:05:54.362223+0000 2024-09-05T01:13:32.412 INFO:teuthology.orchestra.run.smithi129.stderr:listed 10 entries 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 msgr2=0x7f13900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 0x7f13900a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13a0000c00 tx=0x7f13a0031400 comp rx=0 tx=0).stop 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 msgr2=0x7f13a4198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f139800c970 tx=0x7f139800ce30 comp rx=0 tx=0).stop 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 shutdown_connections 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f13900a2b80 0x7f13900a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f13a4106a80 0x7f13a419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f13a4101420 0x7f13a4198860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 --2- 172.21.15.129:0/3044441834 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f13a4100a80 0x7f13a4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 >> 172.21.15.129:0/3044441834 conn(0x7f13a40fc7a0 msgr2=0x7f13a4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 shutdown_connections 2024-09-05T01:13:32.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:32.411+0000 7f13ac1e4640 1 -- 172.21.15.129:0/3044441834 wait complete. 2024-09-05T01:13:33.088 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:32 smithi129 ceph-mon[30979]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:33.088 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:32 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3044441834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:33.116 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-09-05T01:13:33.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:33.117 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-05T01:13:33.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi129.stderr:ceph-fuse[74775]: fuse finished with error 0 and tester_r 0 2024-09-05T01:13:33.146 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T01:13:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:32 smithi183 ceph-mon[34589]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:32 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3044441834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:32 smithi194 ceph-mon[34941]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:13:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:32 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3044441834' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:13:34.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:34 smithi183 ceph-mon[34589]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:34 smithi129 ceph-mon[30979]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:34 smithi194 ceph-mon[34941]: pgmap v294: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:37.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:36 smithi183 ceph-mon[34589]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:36 smithi129 ceph-mon[30979]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:36 smithi194 ceph-mon[34941]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:38 smithi183 ceph-mon[34589]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:38 smithi129 ceph-mon[30979]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:39.248 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:13:39.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:13:39.249 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-05T01:13:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:38 smithi194 ceph-mon[34941]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T01:13:39.279 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:13:39.576 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:40.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 -- 172.21.15.129:0/2396890066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280106f10 msgr2=0x7f528010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:40.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 --2- 172.21.15.129:0/2396890066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280106f10 0x7f528010ea90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5278009ca0 tx=0x7f527802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 -- 172.21.15.129:0/2396890066 shutdown_connections 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 --2- 172.21.15.129:0/2396890066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280106f10 0x7f528010ea90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 --2- 172.21.15.129:0/2396890066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280102790 0x7f52801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 --2- 172.21.15.129:0/2396890066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280101de0 0x7f52801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 -- 172.21.15.129:0/2396890066 >> 172.21.15.129:0/2396890066 conn(0x7f52800fd8c0 msgr2=0x7f52800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.047+0000 7f5286036640 1 -- 172.21.15.129:0/2396890066 shutdown_connections 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 -- 172.21.15.129:0/2396890066 wait complete. 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 Processor -- start 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 -- start start 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280101de0 0x7f5280198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.048+0000 7f5286036640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280106f10 0x7f528019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f5286036640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111e40 con 0x7f5280102790 2024-09-05T01:13:40.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f5286036640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111cc0 con 0x7f5280106f10 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f5286036640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5280111fc0 con 0x7f5280101de0 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50272/0 (socket says 172.21.15.129:50272) 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 -- 172.21.15.129:0/541004012 learned_addr learned my addr 172.21.15.129:0/541004012 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527f7fe640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280101de0 0x7f5280198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 -- 172.21.15.129:0/541004012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280101de0 msgr2=0x7f5280198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280101de0 0x7f5280198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527ffff640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280106f10 0x7f528019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 -- 172.21.15.129:0/541004012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280106f10 msgr2=0x7f528019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280106f10 0x7f528019ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 -- 172.21.15.129:0/541004012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528019d3b0 con 0x7f5280102790 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527effd640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f527400b4a0 tx=0x7f527400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:40.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274014070 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5274010ac0 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528019d6a0 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274017c30 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.049+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a4f80 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.050+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52801021c0 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.051+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5274017070 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.052+0000 7f527cff9640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.052+0000 7f527f7fe640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.052+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f52740ce6e0 con 0x7f5280102790 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.052+0000 7f527f7fe640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 0x7f52680a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f527000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:40.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.056+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f527409b400 con 0x7f5280102790 2024-09-05T01:13:40.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.225+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f528010a150 con 0x7f5280102790 2024-09-05T01:13:40.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.226+0000 7f527cff9640 1 -- 172.21.15.129:0/541004012 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f527409ad00 con 0x7f5280102790 2024-09-05T01:13:40.227 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:40.228 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:13:40.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 msgr2=0x7f52680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:40.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 0x7f52680a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f52700058c0 tx=0x7f527000a4c0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 msgr2=0x7f5280198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:40.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f527400b4a0 tx=0x7f527400b960 comp rx=0 tx=0).stop 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 shutdown_connections 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52680a2b80 0x7f52680a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5280106f10 0x7f528019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5280102790 0x7f5280198940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 --2- 172.21.15.129:0/541004012 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5280101de0 0x7f5280198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 >> 172.21.15.129:0/541004012 conn(0x7f52800fd8c0 msgr2=0x7f52800fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 shutdown_connections 2024-09-05T01:13:40.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:40.230+0000 7f5286036640 1 -- 172.21.15.129:0/541004012 wait complete. 2024-09-05T01:13:41.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:40 smithi183 ceph-mon[34589]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:40 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/541004012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:41.213 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:40 smithi129 ceph-mon[30979]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:41.213 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:40 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/541004012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:41.249 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:13:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:40 smithi194 ceph-mon[34941]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:40 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/541004012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:41.502 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 -- 172.21.15.129:0/3334652347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 msgr2=0x7f672010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 --2- 172.21.15.129:0/3334652347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f672010d950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6718009cd0 tx=0x7f671802a8e0 comp rx=0 tx=0).stop 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 -- 172.21.15.129:0/3334652347 shutdown_connections 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 --2- 172.21.15.129:0/3334652347 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f672010d950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 --2- 172.21.15.129:0/3334652347 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720100a80 0x7f6720100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:41.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 --2- 172.21.15.129:0/3334652347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720106a80 0x7f6720106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 -- 172.21.15.129:0/3334652347 >> 172.21.15.129:0/3334652347 conn(0x7f67200fc7a0 msgr2=0x7f67200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 -- 172.21.15.129:0/3334652347 shutdown_connections 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.984+0000 7f6727484640 1 -- 172.21.15.129:0/3334652347 wait complete. 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 Processor -- start 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 -- start start 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720100a80 0x7f67201982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 0x7f672019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6720110d30 con 0x7f6720101420 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6720110bb0 con 0x7f6720100a80 2024-09-05T01:13:41.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6727484640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6720110eb0 con 0x7f6720106a80 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50294/0 (socket says 172.21.15.129:50294) 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 -- 172.21.15.129:0/1662413246 learned_addr learned my addr 172.21.15.129:0/1662413246 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6725220640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720100a80 0x7f67201982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6725a21640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 0x7f672019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 -- 172.21.15.129:0/1662413246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 msgr2=0x7f672019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 0x7f672019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 -- 172.21.15.129:0/1662413246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720100a80 msgr2=0x7f67201982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720100a80 0x7f67201982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.000 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6724a1f640 1 -- 172.21.15.129:0/1662413246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672019d270 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.985+0000 7f6725a21640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 0x7f672019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f6724a1f640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c890 tx=0x7f671c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c004220 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f671c0043c0 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672019d560 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c004a00 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.986+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67201a4e40 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.987+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6720100ee0 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.988+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f671c004ba0 con 0x7f6720101420 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.988+0000 7f67167fc640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 0x7f670c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.989+0000 7f6725220640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 0x7f670c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:42.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.989+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f671c0cef40 con 0x7f6720101420 2024-09-05T01:13:42.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.989+0000 7f6725220640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 0x7f670c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67100058c0 tx=0x7f671000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:42.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:41.993+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f671c014030 con 0x7f6720101420 2024-09-05T01:13:42.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.161+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6720199620 con 0x7f6720101420 2024-09-05T01:13:42.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.163+0000 7f67167fc640 1 -- 172.21.15.129:0/1662413246 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f6720199620 con 0x7f6720101420 2024-09-05T01:13:42.164 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:13:42.164 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 msgr2=0x7f670c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 0x7f670c0a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67100058c0 tx=0x7f671000a2d0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 msgr2=0x7f6720198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c890 tx=0x7f671c00cd50 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 shutdown_connections 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f670c0a2b80 0x7f670c0a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6720106a80 0x7f672019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6720101420 0x7f6720198800 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 --2- 172.21.15.129:0/1662413246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6720100a80 0x7f67201982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 >> 172.21.15.129:0/1662413246 conn(0x7f67200fc7a0 msgr2=0x7f6720104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 shutdown_connections 2024-09-05T01:13:42.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:42.165+0000 7f6727484640 1 -- 172.21.15.129:0/1662413246 wait complete. 2024-09-05T01:13:42.963 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-05T01:13:43.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:42 smithi183 ceph-mon[34589]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:43.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:42 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1662413246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:42 smithi129 ceph-mon[30979]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:42 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1662413246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:42 smithi194 ceph-mon[34941]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:42 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1662413246' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:13:43.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 -- 172.21.15.129:0/605812863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0106f10 msgr2=0x7f56a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 --2- 172.21.15.129:0/605812863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0106f10 0x7f56a010ea90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5698009ca0 tx=0x7f569802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 -- 172.21.15.129:0/605812863 shutdown_connections 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 --2- 172.21.15.129:0/605812863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0106f10 0x7f56a010ea90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 --2- 172.21.15.129:0/605812863 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0102790 0x7f56a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 --2- 172.21.15.129:0/605812863 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0101de0 0x7f56a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.689+0000 7f56a647a640 1 -- 172.21.15.129:0/605812863 >> 172.21.15.129:0/605812863 conn(0x7f56a00fd8c0 msgr2=0x7f56a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- 172.21.15.129:0/605812863 shutdown_connections 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- 172.21.15.129:0/605812863 wait complete. 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 Processor -- start 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- start start 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:43.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0102790 0x7f56a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 0x7f56a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0111e20 con 0x7f56a0101de0 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0111ca0 con 0x7f56a0106f10 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f56a647a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0111fa0 con 0x7f56a0102790 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f569ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f569ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43036/0 (socket says 172.21.15.129:43036) 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.690+0000 7f569ffff640 1 -- 172.21.15.129:0/1178597742 learned_addr learned my addr 172.21.15.129:0/1178597742 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f5697fff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0102790 0x7f56a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f56a4a17640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 0x7f56a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 -- 172.21.15.129:0/1178597742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0102790 msgr2=0x7f56a0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0102790 0x7f56a0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 -- 172.21.15.129:0/1178597742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 msgr2=0x7f56a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 0x7f56a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:43.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 -- 172.21.15.129:0/1178597742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a01a6720 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f56a4a17640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 0x7f56a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569ffff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f569000c920 tx=0x7f569000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690007c70 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5690007e10 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01a6a10 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690018820 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.691+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a6f50 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.692+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a01021c0 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.693+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f56900189c0 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.694+0000 7f569dffb640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 0x7f568c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.694+0000 7f5697fff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 0x7f568c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.694+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f56900cf3b0 con 0x7f56a0101de0 2024-09-05T01:13:43.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.695+0000 7f5697fff640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 0x7f568c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a0075f80 tx=0x7f568800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:43.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.698+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5690014030 con 0x7f56a0101de0 2024-09-05T01:13:43.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:43.902+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56a002d7c0 con 0x7f56a0101de0 2024-09-05T01:13:44.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.821+0000 7f569dffb640 1 -- 172.21.15.129:0/1178597742 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f569009c0d0 con 0x7f56a0101de0 2024-09-05T01:13:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 msgr2=0x7f568c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 0x7f568c0a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a0075f80 tx=0x7f568800b390 comp rx=0 tx=0).stop 2024-09-05T01:13:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 msgr2=0x7f56a00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f569000c920 tx=0x7f569000cde0 comp rx=0 tx=0).stop 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 shutdown_connections 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f568c0a2b80 0x7f568c0a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56a0106f10 0x7f56a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56a0102790 0x7f56a0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 --2- 172.21.15.129:0/1178597742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56a0101de0 0x7f56a00744f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 >> 172.21.15.129:0/1178597742 conn(0x7f56a00fd8c0 msgr2=0x7f56a00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 shutdown_connections 2024-09-05T01:13:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:44.824+0000 7f56a647a640 1 -- 172.21.15.129:0/1178597742 wait complete. 2024-09-05T01:13:44.883 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:44 smithi129 ceph-mon[30979]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:44.883 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:44 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T01:13:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:44 smithi183 ceph-mon[34589]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:44 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T01:13:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:44 smithi194 ceph-mon[34941]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T01:13:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:44 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T01:13:45.703 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-05T01:13:45.704 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-05T01:13:45.961 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:45.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:45 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T01:13:45.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:45 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:46.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:45 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T01:13:46.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:45 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:45 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1178597742' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T01:13:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:45 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.427+0000 7f481a18f640 1 -- 172.21.15.129:0/2468362232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48141027a0 msgr2=0x7f48141067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 --2- 172.21.15.129:0/2468362232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48141027a0 0x7f48141067f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f47fc009900 tx=0x7f47fc02a410 comp rx=0 tx=0).stop 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 -- 172.21.15.129:0/2468362232 shutdown_connections 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 --2- 172.21.15.129:0/2468362232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814106f20 0x7f481410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 --2- 172.21.15.129:0/2468362232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48141027a0 0x7f48141067f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 --2- 172.21.15.129:0/2468362232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814101df0 0x7f48141021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 -- 172.21.15.129:0/2468362232 >> 172.21.15.129:0/2468362232 conn(0x7f48140fd8f0 msgr2=0x7f48140ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 -- 172.21.15.129:0/2468362232 shutdown_connections 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 -- 172.21.15.129:0/2468362232 wait complete. 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 Processor -- start 2024-09-05T01:13:46.438 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.428+0000 7f481a18f640 1 -- start start 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48141027a0 0x7f4814198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 0x7f481419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4814111e50 con 0x7f4814101df0 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4814111cd0 con 0x7f4814106f20 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4814111fd0 con 0x7f48141027a0 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43058/0 (socket says 172.21.15.129:43058) 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 -- 172.21.15.129:0/1753279957 learned_addr learned my addr 172.21.15.129:0/1753279957 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f4813fff640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 0x7f481419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f480affd640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48141027a0 0x7f4814198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 -- 172.21.15.129:0/1753279957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48141027a0 msgr2=0x7f4814198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48141027a0 0x7f4814198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 -- 172.21.15.129:0/1753279957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 msgr2=0x7f481419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 0x7f481419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 -- 172.21.15.129:0/1753279957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481419d3a0 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f4813fff640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 0x7f481419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48137fe640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f480400c920 tx=0x7f480400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804007c70 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481419d690 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4804007e10 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.429+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804018820 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.430+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141a4f70 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.430+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48141021d0 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.432+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f48040189c0 con 0x7f4814101df0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.432+0000 7f48117fa640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.432+0000 7f480affd640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:46.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.432+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f48040cf480 con 0x7f4814101df0 2024-09-05T01:13:46.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.433+0000 7f480affd640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47fc031440 tx=0x7f47fc0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:46.441 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.436+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4804014030 con 0x7f4814101df0 2024-09-05T01:13:46.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.639+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f481410a100 con 0x7f4814101df0 2024-09-05T01:13:46.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.827+0000 7f48117fa640 1 -- 172.21.15.129:0/1753279957 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f480400de20 con 0x7f4814101df0 2024-09-05T01:13:46.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 msgr2=0x7f48000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47fc031440 tx=0x7f47fc0095f0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 msgr2=0x7f48141983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f480400c920 tx=0x7f480400cde0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 shutdown_connections 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814106f20 0x7f481419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48141027a0 0x7f4814198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 --2- 172.21.15.129:0/1753279957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814101df0 0x7f48141983f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 >> 172.21.15.129:0/1753279957 conn(0x7f48140fd8f0 msgr2=0x7f48140feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 shutdown_connections 2024-09-05T01:13:46.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:46.831+0000 7f481a18f640 1 -- 172.21.15.129:0/1753279957 wait complete. 2024-09-05T01:13:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:46 smithi129 ceph-mon[30979]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T01:13:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:46 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-05T01:13:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:46 smithi183 ceph-mon[34589]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T01:13:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:46 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-05T01:13:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:46 smithi194 ceph-mon[34941]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-05T01:13:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:46 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-05T01:13:47.651 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-05T01:13:47.905 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-05T01:13:47.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:47 smithi129 ceph-mon[30979]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-05T01:13:48.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T01:13:48.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-05T01:13:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T01:13:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-05T01:13:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:47 smithi183 ceph-mon[34589]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1753279957' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-05T01:13:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:47 smithi194 ceph-mon[34941]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-05T01:13:48.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/655445784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc101420 msgr2=0x7f5dbc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:48.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/655445784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc101420 0x7f5dbc10d950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5dac009930 tx=0x7f5dac02a420 comp rx=0 tx=0).stop 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/655445784 shutdown_connections 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/655445784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc101420 0x7f5dbc10d950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/655445784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc100a80 0x7f5dbc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/655445784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/655445784 >> 172.21.15.129:0/655445784 conn(0x7f5dbc0fc7a0 msgr2=0x7f5dbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/655445784 shutdown_connections 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.315+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/655445784 wait complete. 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 Processor -- start 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 -- start start 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 0x7f5dbc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:48.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc110d50 con 0x7f5dbc100a80 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc110bd0 con 0x7f5dbc101420 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dc0ce5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc110ed0 con 0x7f5dbc106a80 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43076/0 (socket says 172.21.15.129:43076) 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.316+0000 7f5dba59c640 1 -- 172.21.15.129:0/3046269910 learned_addr learned my addr 172.21.15.129:0/3046269910 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dbad9d640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5db9d9b640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 0x7f5dbc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 -- 172.21.15.129:0/3046269910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 msgr2=0x7f5dbc19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 -- 172.21.15.129:0/3046269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 msgr2=0x7f5dbc198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 0x7f5dbc198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 -- 172.21.15.129:0/3046269910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc19d2b0 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5db9d9b640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 0x7f5dbc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dbad9d640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dba59c640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5db400ba50 tx=0x7f5db400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4019070 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5db400cbe0 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc19d5a0 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db40075d0 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.317+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc1a4e80 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.318+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc106e60 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.319+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5db401c070 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.320+0000 7f5db37fe640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 0x7f5da80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.321+0000 7f5db9d9b640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 0x7f5da80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.321+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f5db40cedd0 con 0x7f5dbc100a80 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.321+0000 7f5db9d9b640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 0x7f5da80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0febe0 tx=0x7f5da4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:48.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.324+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5db409baf0 con 0x7f5dbc100a80 2024-09-05T01:13:48.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.525+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5dbc1a5160 con 0x7f5dbc100a80 2024-09-05T01:13:48.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.844+0000 7f5db37fe640 1 -- 172.21.15.129:0/3046269910 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f5db409b3f0 con 0x7f5dbc100a80 2024-09-05T01:13:48.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.846+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 msgr2=0x7f5da80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.846+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 0x7f5da80a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc0febe0 tx=0x7f5da4005d90 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.846+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 msgr2=0x7f5dbc198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.846+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5db400ba50 tx=0x7f5db400bf10 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 shutdown_connections 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5da80a2b80 0x7f5da80a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5dbc106a80 0x7f5dbc19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5dbc101420 0x7f5dbc198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 --2- 172.21.15.129:0/3046269910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5dbc100a80 0x7f5dbc198300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 >> 172.21.15.129:0/3046269910 conn(0x7f5dbc0fc7a0 msgr2=0x7f5dbc104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 shutdown_connections 2024-09-05T01:13:48.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:48.847+0000 7f5dc0ce5640 1 -- 172.21.15.129:0/3046269910 wait complete. 2024-09-05T01:13:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-09-05T01:13:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:active 2024-09-05T01:13:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T01:13:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T01:13:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:13:49.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-05T01:13:49.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-05T01:13:49.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T01:13:49.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:48 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T01:13:49.237 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:active 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T01:13:49.238 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 0 op/s 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] up:active 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:13:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-05T01:13:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-05T01:13:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T01:13:49.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:48 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T01:13:49.690 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-05T01:13:49.943 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:49.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:49 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T01:13:49.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:49 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:active 2024-09-05T01:13:49.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:49 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-05T01:13:49.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:49 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:50.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:49 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T01:13:50.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:49 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:active 2024-09-05T01:13:50.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:49 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-05T01:13:50.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:49 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:49 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3046269910' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T01:13:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:49 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] up:active 2024-09-05T01:13:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:49 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-05T01:13:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:49 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:50.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 -- 172.21.15.129:0/1969827536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 msgr2=0x7fa2601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 --2- 172.21.15.129:0/1969827536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601067e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa254009900 tx=0x7fa25402a410 comp rx=0 tx=0).stop 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 -- 172.21.15.129:0/1969827536 shutdown_connections 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 --2- 172.21.15.129:0/1969827536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa260106f10 0x7fa26010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 --2- 172.21.15.129:0/1969827536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601067e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 --2- 172.21.15.129:0/1969827536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 0x7fa2601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 -- 172.21.15.129:0/1969827536 >> 172.21.15.129:0/1969827536 conn(0x7fa2600fd8c0 msgr2=0x7fa2600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 -- 172.21.15.129:0/1969827536 shutdown_connections 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.452+0000 7fa264f97640 1 -- 172.21.15.129:0/1969827536 wait complete. 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 Processor -- start 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 -- start start 2024-09-05T01:13:50.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 0x7fa260198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa260106f10 0x7fa26019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa260111e20 con 0x7fa260102790 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa260111ca0 con 0x7fa260101de0 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa264f97640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa260111fa0 con 0x7fa260106f10 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43094/0 (socket says 172.21.15.129:43094) 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 -- 172.21.15.129:0/1066693287 learned_addr learned my addr 172.21.15.129:0/1066693287 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25e59c640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 0x7fa260198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 -- 172.21.15.129:0/1066693287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa260106f10 msgr2=0x7fa26019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:50.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa260106f10 0x7fa26019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 -- 172.21.15.129:0/1066693287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 msgr2=0x7fa260198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 0x7fa260198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.453+0000 7fa25dd9b640 1 -- 172.21.15.129:0/1066693287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26019d310 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa25dd9b640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa254005940 tx=0x7fa254004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25403f030 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26019d5a0 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa254003aa0 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254035070 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.454+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2601a4ee0 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.455+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2601032c0 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.456+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa254042030 con 0x7fa260102790 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.457+0000 7fa24f7fe640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 0x7fa2480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:50.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.457+0000 7fa25e59c640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 0x7fa2480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:50.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.457+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fa2540efb40 con 0x7fa260102790 2024-09-05T01:13:50.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.457+0000 7fa25e59c640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 0x7fa2480a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa250005fd0 tx=0x7fa25000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:50.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.461+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2540bc800 con 0x7fa260102790 2024-09-05T01:13:50.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.663+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa26010a120 con 0x7fa260102790 2024-09-05T01:13:50.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.855+0000 7fa24f7fe640 1 -- 172.21.15.129:0/1066693287 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fa2540bc100 con 0x7fa260102790 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 msgr2=0x7fa2480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 0x7fa2480a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa250005fd0 tx=0x7fa25000b590 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 msgr2=0x7fa2601988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa254005940 tx=0x7fa254004270 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 shutdown_connections 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa2480a2b80 0x7fa2480a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa260106f10 0x7fa26019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa260102790 0x7fa2601988a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 --2- 172.21.15.129:0/1066693287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa260101de0 0x7fa260198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 >> 172.21.15.129:0/1066693287 conn(0x7fa2600fd8c0 msgr2=0x7fa2600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 shutdown_connections 2024-09-05T01:13:50.859 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:50.857+0000 7fa264f97640 1 -- 172.21.15.129:0/1066693287 wait complete. 2024-09-05T01:13:50.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:50 smithi129 ceph-mon[30979]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-05T01:13:50.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:50 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T01:13:51.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:50 smithi183 ceph-mon[34589]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-05T01:13:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:50 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T01:13:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:50 smithi194 ceph-mon[34941]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 3 op/s 2024-09-05T01:13:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:50 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T01:13:51.692 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-05T01:13:51.692 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-05T01:13:51.944 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:51.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T01:13:51.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:51 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T01:13:52.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:51 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:52.161 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:13:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: 2024-09-05T01:13:51.806+0000 7f8eb1ae0640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-05T01:13:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1066693287' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T01:13:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:51 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:13:52.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 -- 172.21.15.129:0/423717922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4100350 msgr2=0x7f07e4100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:52.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 --2- 172.21.15.129:0/423717922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4100350 0x7f07e4100730 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009930 tx=0x7f07e002a420 comp rx=0 tx=0).stop 2024-09-05T01:13:52.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 -- 172.21.15.129:0/423717922 shutdown_connections 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 --2- 172.21.15.129:0/423717922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4106f50 0x7f07e410eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 --2- 172.21.15.129:0/423717922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 --2- 172.21.15.129:0/423717922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4100350 0x7f07e4100730 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 -- 172.21.15.129:0/423717922 >> 172.21.15.129:0/423717922 conn(0x7f07e4077210 msgr2=0x7f07e40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 -- 172.21.15.129:0/423717922 shutdown_connections 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.422+0000 7f07ebc58640 1 -- 172.21.15.129:0/423717922 wait complete. 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 Processor -- start 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 -- start start 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 0x7f07e4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4111e50 con 0x7f07e4106f50 2024-09-05T01:13:52.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4111cd0 con 0x7f07e4100d00 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ebc58640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07e4111fd0 con 0x7f07e4100350 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ea1f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ea1f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35082/0 (socket says 172.21.15.129:35082) 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ea1f5640 1 -- 172.21.15.129:0/457838710 learned_addr learned my addr 172.21.15.129:0/457838710 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07e91f3640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07e99f4640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 0x7f07e4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.423+0000 7f07ea1f5640 1 -- 172.21.15.129:0/457838710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 msgr2=0x7f07e4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ea1f5640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 0x7f07e4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ea1f5640 1 -- 172.21.15.129:0/457838710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 msgr2=0x7f07e4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ea1f5640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ea1f5640 1 -- 172.21.15.129:0/457838710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e419d3c0 con 0x7f07e4106f50 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07e91f3640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07e99f4640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 0x7f07e4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:13:52.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ea1f5640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00cce0 tx=0x7f07dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00f450 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07dc00faa0 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e419d6b0 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07dc018b70 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.424+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e41a4f90 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.425+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e4100730 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.426+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f07dc018d10 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.427+0000 7f07daffd640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 0x7f07d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.427+0000 7f07e99f4640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 0x7f07d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.427+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f07dc0cf700 con 0x7f07e4106f50 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.427+0000 7f07e99f4640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 0x7f07d00a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:52.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.431+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07dc014030 con 0x7f07e4106f50 2024-09-05T01:13:52.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.617+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f07e4062630 con 0x7f07d00a2b80 2024-09-05T01:13:52.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.642+0000 7f07daffd640 1 -- 172.21.15.129:0/457838710 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f07e4062630 con 0x7f07d00a2b80 2024-09-05T01:13:52.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 msgr2=0x7f07d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:52.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 0x7f07d00a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0031400 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 msgr2=0x7f07e419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f07dc00cce0 tx=0x7f07dc00a620 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 shutdown_connections 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07d00a2b80 0x7f07d00a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07e4106f50 0x7f07e419cce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07e4100d00 0x7f07e4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 --2- 172.21.15.129:0/457838710 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07e4100350 0x7f07e4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 >> 172.21.15.129:0/457838710 conn(0x7f07e4077210 msgr2=0x7f07e41052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 shutdown_connections 2024-09-05T01:13:52.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:52.644+0000 7f07ebc58640 1 -- 172.21.15.129:0/457838710 wait complete. 2024-09-05T01:13:52.869 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:13:52.623+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:13:52.870 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:13:52.623+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:13:52.870 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:13:52.623+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:13:52.870 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:13:52.623+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:13:52.870 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:13:52.623+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:13:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:52 smithi183 ceph-mon[34589]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-09-05T01:13:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:52 smithi129 ceph-mon[30979]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-09-05T01:13:53.561 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:52 smithi194 ceph-mon[34941]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-09-05T01:13:53.562 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-05T01:13:53.815 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:54.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:53 smithi183 ceph-mon[34589]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:54.209 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:53 smithi129 ceph-mon[30979]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:53 smithi194 ceph-mon[34941]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.293+0000 7f1209150640 1 -- 172.21.15.129:0/3123307113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 msgr2=0x7f120410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.293+0000 7f1209150640 1 --2- 172.21.15.129:0/3123307113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f120410d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009cd0 tx=0x7f11fc02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 -- 172.21.15.129:0/3123307113 shutdown_connections 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 --2- 172.21.15.129:0/3123307113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f120410d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 --2- 172.21.15.129:0/3123307113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204100a80 0x7f1204100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 --2- 172.21.15.129:0/3123307113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204106a80 0x7f1204106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 -- 172.21.15.129:0/3123307113 >> 172.21.15.129:0/3123307113 conn(0x7f12040fc7a0 msgr2=0x7f12040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 -- 172.21.15.129:0/3123307113 shutdown_connections 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 -- 172.21.15.129:0/3123307113 wait complete. 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.294+0000 7f1209150640 1 Processor -- start 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 -- start start 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 0x7f12040fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204106a80 0x7f12040fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1204110d30 con 0x7f1204101420 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1204110bb0 con 0x7f1204106a80 2024-09-05T01:13:54.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1209150640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1204110eb0 con 0x7f1204100a80 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35116/0 (socket says 172.21.15.129:35116) 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 -- 172.21.15.129:0/1930098430 learned_addr learned my addr 172.21.15.129:0/1930098430 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120359e640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204106a80 0x7f12040fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1202d9d640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 0x7f12040fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 -- 172.21.15.129:0/1930098430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 msgr2=0x7f12040fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 0x7f12040fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 -- 172.21.15.129:0/1930098430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204106a80 msgr2=0x7f12040fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204106a80 0x7f12040fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 -- 172.21.15.129:0/1930098430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12041a6820 con 0x7f1204101420 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f1202d9d640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 0x7f12040fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.295+0000 7f120259c640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d950 tx=0x7f11f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:54.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.296+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4017070 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.296+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12041a6b10 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.296+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11f4009c30 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.296+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f400f620 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.296+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12041a7050 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.297+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1204100ee0 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.298+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f11f40109a0 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.299+0000 7f11fbfff640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 0x7f11f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.299+0000 7f1202d9d640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 0x7f11f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.299+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f11f40cf040 con 0x7f1204101420 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.299+0000 7f1202d9d640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 0x7f11f00a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005d70 tx=0x7f11ec005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:54.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.303+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11f4009de0 con 0x7f1204101420 2024-09-05T01:13:54.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.494+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1204062630 con 0x7f11f00a2c10 2024-09-05T01:13:54.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.519+0000 7f11fbfff640 1 -- 172.21.15.129:0/1930098430 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1204062630 con 0x7f11f00a2c10 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 msgr2=0x7f11f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 0x7f11f00a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11ec005d70 tx=0x7f11ec005d00 comp rx=0 tx=0).stop 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 msgr2=0x7f12040fd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d950 tx=0x7f11f400de10 comp rx=0 tx=0).stop 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 shutdown_connections 2024-09-05T01:13:54.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f11f00a2c10 0x7f11f00a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1204106a80 0x7f12040fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1204101420 0x7f12040fd390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 --2- 172.21.15.129:0/1930098430 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1204100a80 0x7f12040fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 >> 172.21.15.129:0/1930098430 conn(0x7f12040fc7a0 msgr2=0x7f120406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 shutdown_connections 2024-09-05T01:13:54.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:54.521+0000 7f1209150640 1 -- 172.21.15.129:0/1930098430 wait complete. 2024-09-05T01:13:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:54 smithi183 ceph-mon[34589]: pgmap v304: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T01:13:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:54 smithi183 ceph-mon[34589]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T01:13:55.199 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:54 smithi129 ceph-mon[30979]: pgmap v304: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T01:13:55.199 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:54 smithi129 ceph-mon[30979]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T01:13:55.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-05T01:13:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:54 smithi194 ceph-mon[34941]: pgmap v304: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T01:13:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:54 smithi194 ceph-mon[34941]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-05T01:13:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:55.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 -- 172.21.15.129:0/1240434158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 msgr2=0x7f4b98104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/1240434158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b98104e80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009900 tx=0x7f4b8c02a410 comp rx=0 tx=0).stop 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 -- 172.21.15.129:0/1240434158 shutdown_connections 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/1240434158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 0x7f4b9810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/1240434158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b98104e80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.946+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/1240434158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 0x7f4b98100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 -- 172.21.15.129:0/1240434158 >> 172.21.15.129:0/1240434158 conn(0x7f4b98077210 msgr2=0x7f4b980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 -- 172.21.15.129:0/1240434158 shutdown_connections 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 -- 172.21.15.129:0/1240434158 wait complete. 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 Processor -- start 2024-09-05T01:13:55.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 -- start start 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 0x7f4b980fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 0x7f4b980fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.947+0000 7f4b9dca7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810fc40 con 0x7f4b98100e30 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b9dca7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810fac0 con 0x7f4b981055b0 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b9dca7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810fdc0 con 0x7f4b98100480 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35136/0 (socket says 172.21.15.129:35136) 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 -- 172.21.15.129:0/3186470064 learned_addr learned my addr 172.21.15.129:0/3186470064 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b977fe640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 0x7f4b980fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b97fff640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 0x7f4b980fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:55.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 -- 172.21.15.129:0/3186470064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 msgr2=0x7f4b980fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 0x7f4b980fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 -- 172.21.15.129:0/3186470064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 msgr2=0x7f4b980fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 0x7f4b980fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 -- 172.21.15.129:0/3186470064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b981a6850 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b96ffd640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005940 tx=0x7f4b8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c03f030 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b8c003aa0 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b981a6ae0 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c035070 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.948+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b981a6fc0 con 0x7f4b98100e30 2024-09-05T01:13:55.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.949+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b98100860 con 0x7f4b98100e30 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.950+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4b8c042030 con 0x7f4b98100e30 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.951+0000 7f4b94ff9640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 0x7f4b800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.951+0000 7f4b977fe640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 0x7f4b800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.951+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f4b8c0efb10 con 0x7f4b98100e30 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.952+0000 7f4b977fe640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 0x7f4b800a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007ec0 tx=0x7f4b880075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:55.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:55.955+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4b8c0bc080 con 0x7f4b98100e30 2024-09-05T01:13:56.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.146+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f4b9810a870 con 0x7f4b800a2c10 2024-09-05T01:13:56.147 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:55 smithi129 ceph-mon[30979]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:56.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.152+0000 7f4b94ff9640 1 -- 172.21.15.129:0/3186470064 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f4b8801b4d0 con 0x7f4b800a2c10 2024-09-05T01:13:56.153 INFO:teuthology.orchestra.run.smithi129.stdout:/volumes/qa/sv_0/0fb29cbb-b1e0-4ed3-9e40-e7d25670feba 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 msgr2=0x7f4b800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 0x7f4b800a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4b88007ec0 tx=0x7f4b880075b0 comp rx=0 tx=0).stop 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 msgr2=0x7f4b980fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005940 tx=0x7f4b8c004270 comp rx=0 tx=0).stop 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 shutdown_connections 2024-09-05T01:13:56.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4b800a2c10 0x7f4b800a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b981055b0 0x7f4b980fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b98100e30 0x7f4b980fcbd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 --2- 172.21.15.129:0/3186470064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b98100480 0x7f4b980fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 >> 172.21.15.129:0/3186470064 conn(0x7f4b98077210 msgr2=0x7f4b980fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.154+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 shutdown_connections 2024-09-05T01:13:56.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:56.155+0000 7f4b9dca7640 1 -- 172.21.15.129:0/3186470064 wait complete. 2024-09-05T01:13:56.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:55 smithi183 ceph-mon[34589]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:56.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:55 smithi194 ceph-mon[34941]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:56.862 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-05T01:13:57.107 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:57.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:56 smithi183 ceph-mon[34589]: pgmap v305: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-05T01:13:57.228 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:56 smithi129 ceph-mon[30979]: pgmap v305: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-05T01:13:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:56 smithi194 ceph-mon[34941]: pgmap v305: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-05T01:13:57.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 -- 172.21.15.129:0/3421232282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 msgr2=0x7fe394103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/3421232282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe394103120 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe384009900 tx=0x7fe38402a480 comp rx=0 tx=0).stop 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 -- 172.21.15.129:0/3421232282 shutdown_connections 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/3421232282 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 0x7fe39410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/3421232282 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe39410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/3421232282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe394103120 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 -- 172.21.15.129:0/3421232282 >> 172.21.15.129:0/3421232282 conn(0x7fe3940fc7a0 msgr2=0x7fe3940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 -- 172.21.15.129:0/3421232282 shutdown_connections 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.580+0000 7fe39b2b9640 1 -- 172.21.15.129:0/3421232282 wait complete. 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 Processor -- start 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 -- start start 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:57.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe394100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 0x7fe394101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe394111e20 con 0x7fe394102d40 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe394111ca0 con 0x7fe39410c170 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe39b2b9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe394111fa0 con 0x7fe394103660 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.581+0000 7fe399055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35158/0 (socket says 172.21.15.129:35158) 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 -- 172.21.15.129:0/405027358 learned_addr learned my addr 172.21.15.129:0/405027358 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe398854640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe394100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399856640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 0x7fe394101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 -- 172.21.15.129:0/405027358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 msgr2=0x7fe394100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe394100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 -- 172.21.15.129:0/405027358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 msgr2=0x7fe394101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 0x7fe394101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 -- 172.21.15.129:0/405027358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3941a6810 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe398854640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe394100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe399055640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe384005940 tx=0x7fe3840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38403f030 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3941a6aa0 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe384003cb0 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384035070 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.582+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941a6fd0 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.583+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe394103120 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.584+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe384042030 con 0x7fe394102d40 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.585+0000 7fe38a7fc640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 0x7fe3800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:57.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.585+0000 7fe398854640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 0x7fe3800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:57.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.585+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe3840ef930 con 0x7fe394102d40 2024-09-05T01:13:57.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.585+0000 7fe398854640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 0x7fe3800a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3900058c0 tx=0x7fe390005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:57.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.589+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3840bc4f0 con 0x7fe394102d40 2024-09-05T01:13:57.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.779+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe394062630 con 0x7fe3800a2c10 2024-09-05T01:13:57.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.804+0000 7fe38a7fc640 1 -- 172.21.15.129:0/405027358 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe394062630 con 0x7fe3800a2c10 2024-09-05T01:13:57.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 msgr2=0x7fe3800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:57.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 0x7fe3800a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe3900058c0 tx=0x7fe390005850 comp rx=0 tx=0).stop 2024-09-05T01:13:57.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 msgr2=0x7fe3941009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe384005940 tx=0x7fe3840042f0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 shutdown_connections 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3800a2c10 0x7fe3800a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe39410c170 0x7fe394101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe394103660 0x7fe394100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 --2- 172.21.15.129:0/405027358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe394102d40 0x7fe3941009a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 >> 172.21.15.129:0/405027358 conn(0x7fe3940fc7a0 msgr2=0x7fe3940fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 shutdown_connections 2024-09-05T01:13:57.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:57.806+0000 7fe39b2b9640 1 -- 172.21.15.129:0/405027358 wait complete. 2024-09-05T01:13:58.117 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:57 smithi129 ceph-mon[30979]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:58.151 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:57 smithi194 ceph-mon[34941]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:58.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:57 smithi183 ceph-mon[34589]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:58.431 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-05T01:13:58.677 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:13:59.093 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:58 smithi129 ceph-mon[30979]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-05T01:13:59.093 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:13:58 smithi129 ceph-mon[30979]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:59.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:58 smithi183 ceph-mon[34589]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-05T01:13:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:13:58 smithi183 ceph-mon[34589]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 -- 172.21.15.129:0/2460363665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb41082a0 msgr2=0x7f1eb410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 --2- 172.21.15.129:0/2460363665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb41082a0 0x7f1eb410ec40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1eac009cd0 tx=0x7f1eac02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 -- 172.21.15.129:0/2460363665 shutdown_connections 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 --2- 172.21.15.129:0/2460363665 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb41082a0 0x7f1eb410ec40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 --2- 172.21.15.129:0/2460363665 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1eb41036e0 0x7f1eb4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 --2- 172.21.15.129:0/2460363665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb4102d30 0x7f1eb4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 -- 172.21.15.129:0/2460363665 >> 172.21.15.129:0/2460363665 conn(0x7f1eb40fc7d0 msgr2=0x7f1eb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 -- 172.21.15.129:0/2460363665 shutdown_connections 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.176+0000 7f1eb994b640 1 -- 172.21.15.129:0/2460363665 wait complete. 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 Processor -- start 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 -- start start 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb41036e0 0x7f1eb41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1eb41082a0 0x7f1eb419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4111f50 con 0x7f1eb4102d30 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb4111dd0 con 0x7f1eb41036e0 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb994b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb41120d0 con 0x7f1eb41082a0 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35176/0 (socket says 172.21.15.129:35176) 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.177+0000 7f1eb2ffd640 1 -- 172.21.15.129:0/3167963512 learned_addr learned my addr 172.21.15.129:0/3167963512 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 -- 172.21.15.129:0/3167963512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1eb41082a0 msgr2=0x7f1eb419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb27fc640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb41036e0 0x7f1eb41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1eb41082a0 0x7f1eb419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 -- 172.21.15.129:0/3167963512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb41036e0 msgr2=0x7f1eb41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb41036e0 0x7f1eb41989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 -- 172.21.15.129:0/3167963512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb419d460 con 0x7f1eb4102d30 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb2ffd640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c890 tx=0x7f1ea400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004220 con 0x7f1eb4102d30 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ea40043c0 con 0x7f1eb4102d30 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb419d750 con 0x7f1eb4102d30 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea4004a00 con 0x7f1eb4102d30 2024-09-05T01:13:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.178+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb41a5030 con 0x7f1eb4102d30 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.179+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb4103110 con 0x7f1eb4102d30 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.180+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1ea4004ba0 con 0x7f1eb4102d30 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.181+0000 7f1eb8949640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 0x7f1e9c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.181+0000 7f1eb27fc640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 0x7f1e9c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.181+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f1ea40cf160 con 0x7f1eb4102d30 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.181+0000 7f1eb27fc640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 0x7f1e9c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8005d80 tx=0x7f1ea8005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:13:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.185+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ea4014030 con 0x7f1eb4102d30 2024-09-05T01:13:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:58 smithi194 ceph-mon[34941]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-05T01:13:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:13:58 smithi194 ceph-mon[34941]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:13:59.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.377+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1eb4062630 con 0x7f1e9c0a2c90 2024-09-05T01:13:59.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.382+0000 7f1eb8949640 1 -- 172.21.15.129:0/3167963512 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1eb4062630 con 0x7f1e9c0a2c90 2024-09-05T01:13:59.383 INFO:teuthology.orchestra.run.smithi129.stdout:/volumes/qa/sv_1/360ae03b-8d70-4498-84ef-a2f227b53ae7 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 msgr2=0x7f1e9c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 0x7f1e9c0a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8005d80 tx=0x7f1ea8005c90 comp rx=0 tx=0).stop 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 msgr2=0x7f1eb41984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400c890 tx=0x7f1ea400cd50 comp rx=0 tx=0).stop 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 shutdown_connections 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e9c0a2c90 0x7f1e9c0a5150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1eb41082a0 0x7f1eb419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1eb41036e0 0x7f1eb41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 --2- 172.21.15.129:0/3167963512 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1eb4102d30 0x7f1eb41984b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:13:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 >> 172.21.15.129:0/3167963512 conn(0x7f1eb40fc7d0 msgr2=0x7f1eb4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:13:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 shutdown_connections 2024-09-05T01:13:59.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:13:59.384+0000 7f1eb994b640 1 -- 172.21.15.129:0/3167963512 wait complete. 2024-09-05T01:14:00.086 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:14:00.393 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:00.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 -- 172.21.15.129:0/1666113141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 msgr2=0x7f056410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:00.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 --2- 172.21.15.129:0/1666113141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056410e610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0554009900 tx=0x7f055402a480 comp rx=0 tx=0).stop 2024-09-05T01:14:00.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 -- 172.21.15.129:0/1666113141 shutdown_connections 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 --2- 172.21.15.129:0/1666113141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056410e610 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 --2- 172.21.15.129:0/1666113141 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 0x7f056410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 --2- 172.21.15.129:0/1666113141 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f0564103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 -- 172.21.15.129:0/1666113141 >> 172.21.15.129:0/1666113141 conn(0x7f05640fc7a0 msgr2=0x7f05640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 -- 172.21.15.129:0/1666113141 shutdown_connections 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 -- 172.21.15.129:0/1666113141 wait complete. 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.846+0000 7f05694a9640 1 Processor -- start 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 -- start start 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:00.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 0x7f0564198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111e20 con 0x7f056410c170 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111ca0 con 0x7f0564102d40 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05694a9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0564111fa0 con 0x7f0564103660 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f0562ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f0562ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:36738/0 (socket says 172.21.15.129:36738) 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f0562ffd640 1 -- 172.21.15.129:0/825930797 learned_addr learned my addr 172.21.15.129:0/825930797 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.847+0000 7f05627fc640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 0x7f0564198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:00.856 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 -- 172.21.15.129:0/825930797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 msgr2=0x7f0564198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 0x7f0564198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 -- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 msgr2=0x7f056419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 -- 172.21.15.129:0/825930797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056419d290 con 0x7f0564102d40 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f05637fe640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f0562ffd640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ebc0 tx=0x7f055c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c010630 con 0x7f0564102d40 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f055c010c80 con 0x7f0564102d40 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c018b30 con 0x7f0564102d40 2024-09-05T01:14:00.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056419d580 con 0x7f0564102d40 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.848+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641008d0 con 0x7f0564102d40 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.849+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056406b070 con 0x7f0564102d40 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.852+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f055c018cd0 con 0x7f0564102d40 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.853+0000 7f055bfff640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 0x7f05500a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.853+0000 7f05627fc640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 0x7f05500a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.853+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f055c0cfd90 con 0x7f0564102d40 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.853+0000 7f05627fc640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 0x7f05500a5220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05640feda0 tx=0x7f054c00b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:00.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:00.855+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f055c09c380 con 0x7f0564102d40 2024-09-05T01:14:01.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.077+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0564103120 con 0x7f0564102d40 2024-09-05T01:14:01.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.078+0000 7f055bfff640 1 -- 172.21.15.129:0/825930797 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f055c0a0680 con 0x7f0564102d40 2024-09-05T01:14:01.080 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:01.080 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:14:01.080 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 msgr2=0x7f05500a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 0x7f05500a5220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05640feda0 tx=0x7f054c00b360 comp rx=0 tx=0).stop 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 msgr2=0x7f05641982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f055c00ebc0 tx=0x7f055c00c6a0 comp rx=0 tx=0).stop 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 shutdown_connections 2024-09-05T01:14:01.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f05500a2d60 0x7f05500a5220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f056410c170 0x7f056419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0564103660 0x7f0564198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 --2- 172.21.15.129:0/825930797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0564102d40 0x7f05641982e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 >> 172.21.15.129:0/825930797 conn(0x7f05640fc7a0 msgr2=0x7f05640fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 shutdown_connections 2024-09-05T01:14:01.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:01.082+0000 7f05694a9640 1 -- 172.21.15.129:0/825930797 wait complete. 2024-09-05T01:14:01.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:00 smithi183 ceph-mon[34589]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:14:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:00 smithi183 ceph-mon[34589]: pgmap v307: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-09-05T01:14:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:00 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:14:01.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:00 smithi129 ceph-mon[30979]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:14:01.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:00 smithi129 ceph-mon[30979]: pgmap v307: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-09-05T01:14:01.245 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:00 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:14:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:00 smithi194 ceph-mon[34941]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:14:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:00 smithi194 ceph-mon[34941]: pgmap v307: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2024-09-05T01:14:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:00 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:14:01.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:14:02.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:01 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/825930797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:02.241 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:01 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/825930797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:02.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:01 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/825930797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:02.639 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:03.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:02 smithi129 ceph-mon[30979]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-05T01:14:03.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:02 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:02 smithi183 ceph-mon[34589]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-05T01:14:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:02 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- 172.21.15.129:0/3730282574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 msgr2=0x7fd8ec10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:03.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 --2- 172.21.15.129:0/3730282574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec10d940 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4009cd0 tx=0x7fd8e402a8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- 172.21.15.129:0/3730282574 shutdown_connections 2024-09-05T01:14:03.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 --2- 172.21.15.129:0/3730282574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec10d940 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 --2- 172.21.15.129:0/3730282574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 0x7fd8ec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 --2- 172.21.15.129:0/3730282574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- 172.21.15.129:0/3730282574 >> 172.21.15.129:0/3730282574 conn(0x7fd8ec0fc790 msgr2=0x7fd8ec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- 172.21.15.129:0/3730282574 shutdown_connections 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- 172.21.15.129:0/3730282574 wait complete. 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 Processor -- start 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.150+0000 7fd8f3943640 1 -- start start 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 0x7fd8ec198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec110d20 con 0x7fd8ec101410 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec110ba0 con 0x7fd8ec106a70 2024-09-05T01:14:03.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec110ea0 con 0x7fd8ec100a70 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33080/0 (socket says 172.21.15.129:33080) 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 -- 172.21.15.129:0/297478061 learned_addr learned my addr 172.21.15.129:0/297478061 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f16df640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 0x7fd8ec198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f1ee0640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 -- 172.21.15.129:0/297478061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 msgr2=0x7fd8ec198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 0x7fd8ec198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 -- 172.21.15.129:0/297478061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 msgr2=0x7fd8ec19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 -- 172.21.15.129:0/297478061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec19d2f0 con 0x7fd8ec101410 2024-09-05T01:14:03.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f1ee0640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f0ede640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d950 tx=0x7fd8e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8017030 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8e8004020 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec19d5e0 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80047a0 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.151+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec1a4ec0 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.152+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec100ed0 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.153+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fd8e80041d0 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.154+0000 7fd8de7fc640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 0x7fd8d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.154+0000 7fd8f16df640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 0x7fd8d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.154+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd8e80cee60 con 0x7fd8ec101410 2024-09-05T01:14:03.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.155+0000 7fd8f16df640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 0x7fd8d80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009b50 tx=0x7fd8e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:03.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.158+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8e809ba20 con 0x7fd8ec101410 2024-09-05T01:14:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:02 smithi194 ceph-mon[34941]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.2 KiB/s wr, 11 op/s 2024-09-05T01:14:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:02 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:03.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.354+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8ec1996a0 con 0x7fd8ec101410 2024-09-05T01:14:03.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.356+0000 7fd8de7fc640 1 -- 172.21.15.129:0/297478061 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7fd8ec1996a0 con 0x7fd8ec101410 2024-09-05T01:14:03.357 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:03.357 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:14:03.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.358+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 msgr2=0x7fd8d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:03.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.358+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 0x7fd8d80a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009b50 tx=0x7fd8e0005f90 comp rx=0 tx=0).stop 2024-09-05T01:14:03.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.358+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 msgr2=0x7fd8ec198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:03.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.358+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800d950 tx=0x7fd8e800de10 comp rx=0 tx=0).stop 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 shutdown_connections 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8d80a2c10 0x7fd8d80a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8ec106a70 0x7fd8ec19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8ec101410 0x7fd8ec198880 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 --2- 172.21.15.129:0/297478061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd8ec100a70 0x7fd8ec198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 >> 172.21.15.129:0/297478061 conn(0x7fd8ec0fc790 msgr2=0x7fd8ec104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 shutdown_connections 2024-09-05T01:14:03.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:03.359+0000 7fd8f3943640 1 -- 172.21.15.129:0/297478061 wait complete. 2024-09-05T01:14:04.052 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:03 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/297478061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:04.054 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T01:14:04.055 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:14:04.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:03 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/297478061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:04.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:03 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/297478061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:04.315 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:04.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1449298454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0102d40 msgr2=0x7fe4d0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:04.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1449298454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0102d40 0x7fe4d0103120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002a480 comp rx=0 tx=0).stop 2024-09-05T01:14:04.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1449298454 shutdown_connections 2024-09-05T01:14:04.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1449298454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d010c170 0x7fe4d010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1449298454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d0103660 0x7fe4d010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1449298454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0102d40 0x7fe4d0103120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1449298454 >> 172.21.15.129:0/1449298454 conn(0x7fe4d00fc7a0 msgr2=0x7fe4d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.823+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1449298454 shutdown_connections 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1449298454 wait complete. 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 Processor -- start 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 -- start start 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d0102d40 0x7fe4d0198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d010c170 0x7fe4d019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0111e20 con 0x7fe4d0103660 2024-09-05T01:14:04.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0111ca0 con 0x7fe4d0102d40 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d6e97640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0111fa0 con 0x7fe4d010c170 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33100/0 (socket says 172.21.15.129:33100) 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4cffff640 1 -- 172.21.15.129:0/1531067872 learned_addr learned my addr 172.21.15.129:0/1531067872 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.824+0000 7fe4d4c33640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d0102d40 0x7fe4d0198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4d5434640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d010c170 0x7fe4d019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 -- 172.21.15.129:0/1531067872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d010c170 msgr2=0x7fe4d019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d010c170 0x7fe4d019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 -- 172.21.15.129:0/1531067872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d0102d40 msgr2=0x7fe4d0198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d0102d40 0x7fe4d0198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 -- 172.21.15.129:0/1531067872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d019d310 con 0x7fe4d0103660 2024-09-05T01:14:04.834 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cffff640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d750 tx=0x7fe4c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4017070 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c4005020 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d019d600 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004c60 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.825+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a4ee0 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.826+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d0103120 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.828+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe4c4009a30 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.829+0000 7fe4cdffb640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 0x7fe4b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.829+0000 7fe4d4c33640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 0x7fe4b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.829+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fe4c40cedf0 con 0x7fe4d0103660 2024-09-05T01:14:04.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.829+0000 7fe4d4c33640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 0x7fe4b80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0031440 tx=0x7fe4c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:04.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:04.835+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4c409b950 con 0x7fe4d0103660 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:14:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:04 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.003+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4d0062630 con 0x7fe4b80a2c10 2024-09-05T01:14:05.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.012+0000 7fe4cdffb640 1 -- 172.21.15.129:0/1531067872 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340341 (secure 0 0 0) 0x7fe4d0062630 con 0x7fe4b80a2c10 2024-09-05T01:14:05.013 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:05.020 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.015+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 msgr2=0x7fe4b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.015+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 0x7fe4b80a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0031440 tx=0x7fe4c0005b50 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.015+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 msgr2=0x7fe4d01988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.015+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400d750 tx=0x7fe4c400dc10 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 shutdown_connections 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b80a2c10 0x7fe4b80a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4d010c170 0x7fe4d019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4d0103660 0x7fe4d01988a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 --2- 172.21.15.129:0/1531067872 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4d0102d40 0x7fe4d0198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:05.023 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 >> 172.21.15.129:0/1531067872 conn(0x7fe4d00fc7a0 msgr2=0x7fe4d00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:05.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 shutdown_connections 2024-09-05T01:14:05.024 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:05.016+0000 7fe4d6e97640 1 -- 172.21.15.129:0/1531067872 wait complete. 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:14:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:04 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: pgmap v309: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-09-05T01:14:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:14:05.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:14:05.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:04 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:14:05.659 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-05T01:14:03.760367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647013,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":225,"num_write":338,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387588,"kb_used_data":13380,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554236,"statfs":{"total":1151940427776,"available":1151543537664,"internally_reserved":0,"allocated":13701120,"data_stored":7627190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50619,"num_objects":20,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":110,"num_read_kb":116,"num_write":61,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003259"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.855040+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:13:47.855040+0000","last_peered":"2024-09-05T01:13:47.855040+0000","last_clean":"2024-09-05T01:13:47.855040+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:13:47.855040+0000","last_undegraded":"2024-09-05T01:13:47.855040+0000","last_fullsized":"2024-09-05T01:13:47.855040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911634+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:13:15.911634+0000","last_peered":"2024-09-05T01:13:15.911634+0000","last_clean":"2024-09-05T01:13:15.911634+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:13:15.911634+0000","last_undegraded":"2024-09-05T01:13:15.911634+0000","last_fullsized":"2024-09-05T01:13:15.911634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054088+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:13:16.054088+0000","last_peered":"2024-09-05T01:13:16.054088+0000","last_clean":"2024-09-05T01:13:16.054088+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:13:16.054088+0000","last_undegraded":"2024-09-05T01:13:16.054088+0000","last_fullsized":"2024-09-05T01:13:16.054088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911825+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:13:15.911825+0000","last_peered":"2024-09-05T01:13:15.911825+0000","last_clean":"2024-09-05T01:13:15.911825+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:13:15.911825+0000","last_undegraded":"2024-09-05T01:13:15.911825+0000","last_fullsized":"2024-09-05T01:13:15.911825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054671+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:13:16.054671+0000","last_peered":"2024-09-05T01:13:16.054671+0000","last_clean":"2024-09-05T01:13:16.054671+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:13:16.054671+0000","last_undegraded":"2024-09-05T01:13:16.054671+0000","last_fullsized":"2024-09-05T01:13:16.054671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912648+0000","last_change":"2024-09-05T01:13:12.925330+0000","last_active":"2024-09-05T01:13:15.912648+0000","last_peered":"2024-09-05T01:13:15.912648+0000","last_clean":"2024-09-05T01:13:15.912648+0000","last_became_active":"2024-09-05T01:13:12.924809+0000","last_became_peered":"2024-09-05T01:13:12.924809+0000","last_unstale":"2024-09-05T01:13:15.912648+0000","last_undegraded":"2024-09-05T01:13:15.912648+0000","last_fullsized":"2024-09-05T01:13:15.912648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:18:10.430813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854328+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:13:47.854328+0000","last_peered":"2024-09-05T01:13:47.854328+0000","last_clean":"2024-09-05T01:13:47.854328+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:13:47.854328+0000","last_undegraded":"2024-09-05T01:13:47.854328+0000","last_fullsized":"2024-09-05T01:13:47.854328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055014+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:13:16.055014+0000","last_peered":"2024-09-05T01:13:16.055014+0000","last_clean":"2024-09-05T01:13:16.055014+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:13:16.055014+0000","last_undegraded":"2024-09-05T01:13:16.055014+0000","last_fullsized":"2024-09-05T01:13:16.055014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913036+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:13:15.913036+0000","last_peered":"2024-09-05T01:13:15.913036+0000","last_clean":"2024-09-05T01:13:15.913036+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:13:15.913036+0000","last_undegraded":"2024-09-05T01:13:15.913036+0000","last_fullsized":"2024-09-05T01:13:15.913036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054769+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:13:16.054769+0000","last_peered":"2024-09-05T01:13:16.054769+0000","last_clean":"2024-09-05T01:13:16.054769+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:13:16.054769+0000","last_undegraded":"2024-09-05T01:13:16.054769+0000","last_fullsized":"2024-09-05T01:13:16.054769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912337+0000","last_change":"2024-09-05T01:13:05.901886+0000","last_active":"2024-09-05T01:13:15.912337+0000","last_peered":"2024-09-05T01:13:15.912337+0000","last_clean":"2024-09-05T01:13:15.912337+0000","last_became_active":"2024-09-05T01:13:05.901565+0000","last_became_peered":"2024-09-05T01:13:05.901565+0000","last_unstale":"2024-09-05T01:13:15.912337+0000","last_undegraded":"2024-09-05T01:13:15.912337+0000","last_fullsized":"2024-09-05T01:13:15.912337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:50:21.707874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912764+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:13:15.912764+0000","last_peered":"2024-09-05T01:13:15.912764+0000","last_clean":"2024-09-05T01:13:15.912764+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:13:15.912764+0000","last_undegraded":"2024-09-05T01:13:15.912764+0000","last_fullsized":"2024-09-05T01:13:15.912764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054678+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:13:16.054678+0000","last_peered":"2024-09-05T01:13:16.054678+0000","last_clean":"2024-09-05T01:13:16.054678+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:13:16.054678+0000","last_undegraded":"2024-09-05T01:13:16.054678+0000","last_fullsized":"2024-09-05T01:13:16.054678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845103+0000","last_change":"2024-09-05T01:13:03.882357+0000","last_active":"2024-09-05T01:13:46.845103+0000","last_peered":"2024-09-05T01:13:46.845103+0000","last_clean":"2024-09-05T01:13:46.845103+0000","last_became_active":"2024-09-05T01:13:03.881983+0000","last_became_peered":"2024-09-05T01:13:03.881983+0000","last_unstale":"2024-09-05T01:13:46.845103+0000","last_undegraded":"2024-09-05T01:13:46.845103+0000","last_fullsized":"2024-09-05T01:13:46.845103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:09:44.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913310+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:13:15.913310+0000","last_peered":"2024-09-05T01:13:15.913310+0000","last_clean":"2024-09-05T01:13:15.913310+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:13:15.913310+0000","last_undegraded":"2024-09-05T01:13:15.913310+0000","last_fullsized":"2024-09-05T01:13:15.913310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911262+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:13:15.911262+0000","last_peered":"2024-09-05T01:13:15.911262+0000","last_clean":"2024-09-05T01:13:15.911262+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:13:15.911262+0000","last_undegraded":"2024-09-05T01:13:15.911262+0000","last_fullsized":"2024-09-05T01:13:15.911262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054655+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:13:16.054655+0000","last_peered":"2024-09-05T01:13:16.054655+0000","last_clean":"2024-09-05T01:13:16.054655+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:13:16.054655+0000","last_undegraded":"2024-09-05T01:13:16.054655+0000","last_fullsized":"2024-09-05T01:13:16.054655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.863054+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:13:16.863054+0000","last_peered":"2024-09-05T01:13:16.863054+0000","last_clean":"2024-09-05T01:13:16.863054+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:13:16.863054+0000","last_undegraded":"2024-09-05T01:13:16.863054+0000","last_fullsized":"2024-09-05T01:13:16.863054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424775+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:13:16.424775+0000","last_peered":"2024-09-05T01:13:16.424775+0000","last_clean":"2024-09-05T01:13:16.424775+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:13:16.424775+0000","last_undegraded":"2024-09-05T01:13:16.424775+0000","last_fullsized":"2024-09-05T01:13:16.424775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.843904+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:13:46.843904+0000","last_peered":"2024-09-05T01:13:46.843904+0000","last_clean":"2024-09-05T01:13:46.843904+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:13:46.843904+0000","last_undegraded":"2024-09-05T01:13:46.843904+0000","last_fullsized":"2024-09-05T01:13:46.843904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055534+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:13:16.055534+0000","last_peered":"2024-09-05T01:13:16.055534+0000","last_clean":"2024-09-05T01:13:16.055534+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:13:16.055534+0000","last_undegraded":"2024-09-05T01:13:16.055534+0000","last_fullsized":"2024-09-05T01:13:16.055534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054171+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:13:16.054171+0000","last_peered":"2024-09-05T01:13:16.054171+0000","last_clean":"2024-09-05T01:13:16.054171+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:13:16.054171+0000","last_undegraded":"2024-09-05T01:13:16.054171+0000","last_fullsized":"2024-09-05T01:13:16.054171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.844682+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:13:46.844682+0000","last_peered":"2024-09-05T01:13:46.844682+0000","last_clean":"2024-09-05T01:13:46.844682+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:13:46.844682+0000","last_undegraded":"2024-09-05T01:13:46.844682+0000","last_fullsized":"2024-09-05T01:13:46.844682+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913148+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:13:15.913148+0000","last_peered":"2024-09-05T01:13:15.913148+0000","last_clean":"2024-09-05T01:13:15.913148+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:13:15.913148+0000","last_undegraded":"2024-09-05T01:13:15.913148+0000","last_fullsized":"2024-09-05T01:13:15.913148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054551+0000","last_change":"2024-09-05T01:13:05.882797+0000","last_active":"2024-09-05T01:13:16.054551+0000","last_peered":"2024-09-05T01:13:16.054551+0000","last_clean":"2024-09-05T01:13:16.054551+0000","last_became_active":"2024-09-05T01:13:05.882461+0000","last_became_peered":"2024-09-05T01:13:05.882461+0000","last_unstale":"2024-09-05T01:13:16.054551+0000","last_undegraded":"2024-09-05T01:13:16.054551+0000","last_fullsized":"2024-09-05T01:13:16.054551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:30:00.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845406+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:13:46.845406+0000","last_peered":"2024-09-05T01:13:46.845406+0000","last_clean":"2024-09-05T01:13:46.845406+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:13:46.845406+0000","last_undegraded":"2024-09-05T01:13:46.845406+0000","last_fullsized":"2024-09-05T01:13:46.845406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054097+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:13:16.054097+0000","last_peered":"2024-09-05T01:13:16.054097+0000","last_clean":"2024-09-05T01:13:16.054097+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:13:16.054097+0000","last_undegraded":"2024-09-05T01:13:16.054097+0000","last_fullsized":"2024-09-05T01:13:16.054097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911790+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:13:15.911790+0000","last_peered":"2024-09-05T01:13:15.911790+0000","last_clean":"2024-09-05T01:13:15.911790+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:13:15.911790+0000","last_undegraded":"2024-09-05T01:13:15.911790+0000","last_fullsized":"2024-09-05T01:13:15.911790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'7","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:56.898556+0000","last_change":"2024-09-05T01:13:03.873898+0000","last_active":"2024-09-05T01:13:56.898556+0000","last_peered":"2024-09-05T01:13:56.898556+0000","last_clean":"2024-09-05T01:13:56.898556+0000","last_became_active":"2024-09-05T01:13:03.873546+0000","last_became_peered":"2024-09-05T01:13:03.873546+0000","last_unstale":"2024-09-05T01:13:56.898556+0000","last_undegraded":"2024-09-05T01:13:56.898556+0000","last_fullsized":"2024-09-05T01:13:56.898556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:45:48.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911014+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:13:15.911014+0000","last_peered":"2024-09-05T01:13:15.911014+0000","last_clean":"2024-09-05T01:13:15.911014+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:13:15.911014+0000","last_undegraded":"2024-09-05T01:13:15.911014+0000","last_fullsized":"2024-09-05T01:13:15.911014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054830+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:13:16.054830+0000","last_peered":"2024-09-05T01:13:16.054830+0000","last_clean":"2024-09-05T01:13:16.054830+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:13:16.054830+0000","last_undegraded":"2024-09-05T01:13:16.054830+0000","last_fullsized":"2024-09-05T01:13:16.054830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.843697+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:13:46.843697+0000","last_peered":"2024-09-05T01:13:46.843697+0000","last_clean":"2024-09-05T01:13:46.843697+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:13:46.843697+0000","last_undegraded":"2024-09-05T01:13:46.843697+0000","last_fullsized":"2024-09-05T01:13:46.843697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424812+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:13:16.424812+0000","last_peered":"2024-09-05T01:13:16.424812+0000","last_clean":"2024-09-05T01:13:16.424812+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:13:16.424812+0000","last_undegraded":"2024-09-05T01:13:16.424812+0000","last_fullsized":"2024-09-05T01:13:16.424812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.056018+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:13:16.056018+0000","last_peered":"2024-09-05T01:13:16.056018+0000","last_clean":"2024-09-05T01:13:16.056018+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:13:16.056018+0000","last_undegraded":"2024-09-05T01:13:16.056018+0000","last_fullsized":"2024-09-05T01:13:16.056018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055406+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:13:16.055406+0000","last_peered":"2024-09-05T01:13:16.055406+0000","last_clean":"2024-09-05T01:13:16.055406+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:13:16.055406+0000","last_undegraded":"2024-09-05T01:13:16.055406+0000","last_fullsized":"2024-09-05T01:13:16.055406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054345+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:13:16.054345+0000","last_peered":"2024-09-05T01:13:16.054345+0000","last_clean":"2024-09-05T01:13:16.054345+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:13:16.054345+0000","last_undegraded":"2024-09-05T01:13:16.054345+0000","last_fullsized":"2024-09-05T01:13:16.054345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055644+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:13:16.055644+0000","last_peered":"2024-09-05T01:13:16.055644+0000","last_clean":"2024-09-05T01:13:16.055644+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:13:16.055644+0000","last_undegraded":"2024-09-05T01:13:16.055644+0000","last_fullsized":"2024-09-05T01:13:16.055644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054834+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:13:16.054834+0000","last_peered":"2024-09-05T01:13:16.054834+0000","last_clean":"2024-09-05T01:13:16.054834+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:13:16.054834+0000","last_undegraded":"2024-09-05T01:13:16.054834+0000","last_fullsized":"2024-09-05T01:13:16.054834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054600+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:13:16.054600+0000","last_peered":"2024-09-05T01:13:16.054600+0000","last_clean":"2024-09-05T01:13:16.054600+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:13:16.054600+0000","last_undegraded":"2024-09-05T01:13:16.054600+0000","last_fullsized":"2024-09-05T01:13:16.054600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911766+0000","last_change":"2024-09-05T01:13:05.897936+0000","last_active":"2024-09-05T01:13:15.911766+0000","last_peered":"2024-09-05T01:13:15.911766+0000","last_clean":"2024-09-05T01:13:15.911766+0000","last_became_active":"2024-09-05T01:13:05.897610+0000","last_became_peered":"2024-09-05T01:13:05.897610+0000","last_unstale":"2024-09-05T01:13:15.911766+0000","last_undegraded":"2024-09-05T01:13:15.911766+0000","last_fullsized":"2024-09-05T01:13:15.911766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:14.754999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:01.875161+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:14:01.875161+0000","last_peered":"2024-09-05T01:14:01.875161+0000","last_clean":"2024-09-05T01:14:01.875161+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:14:01.875161+0000","last_undegraded":"2024-09-05T01:14:01.875161+0000","last_fullsized":"2024-09-05T01:14:01.875161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055254+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:13:16.055254+0000","last_peered":"2024-09-05T01:13:16.055254+0000","last_clean":"2024-09-05T01:13:16.055254+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:13:16.055254+0000","last_undegraded":"2024-09-05T01:13:16.055254+0000","last_fullsized":"2024-09-05T01:13:16.055254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.053984+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:13:16.053984+0000","last_peered":"2024-09-05T01:13:16.053984+0000","last_clean":"2024-09-05T01:13:16.053984+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:13:16.053984+0000","last_undegraded":"2024-09-05T01:13:16.053984+0000","last_fullsized":"2024-09-05T01:13:16.053984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854613+0000","last_change":"2024-09-05T01:13:03.880884+0000","last_active":"2024-09-05T01:13:47.854613+0000","last_peered":"2024-09-05T01:13:47.854613+0000","last_clean":"2024-09-05T01:13:47.854613+0000","last_became_active":"2024-09-05T01:13:03.880274+0000","last_became_peered":"2024-09-05T01:13:03.880274+0000","last_unstale":"2024-09-05T01:13:47.854613+0000","last_undegraded":"2024-09-05T01:13:47.854613+0000","last_fullsized":"2024-09-05T01:13:47.854613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:24.973934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912322+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:13:15.912322+0000","last_peered":"2024-09-05T01:13:15.912322+0000","last_clean":"2024-09-05T01:13:15.912322+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:13:15.912322+0000","last_undegraded":"2024-09-05T01:13:15.912322+0000","last_fullsized":"2024-09-05T01:13:15.912322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911162+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:13:15.911162+0000","last_peered":"2024-09-05T01:13:15.911162+0000","last_clean":"2024-09-05T01:13:15.911162+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:13:15.911162+0000","last_undegraded":"2024-09-05T01:13:15.911162+0000","last_fullsized":"2024-09-05T01:13:15.911162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:48.853786+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:13:48.853786+0000","last_peered":"2024-09-05T01:13:48.853786+0000","last_clean":"2024-09-05T01:13:48.853786+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:13:48.853786+0000","last_undegraded":"2024-09-05T01:13:48.853786+0000","last_fullsized":"2024-09-05T01:13:48.853786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054460+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:13:16.054460+0000","last_peered":"2024-09-05T01:13:16.054460+0000","last_clean":"2024-09-05T01:13:16.054460+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:13:16.054460+0000","last_undegraded":"2024-09-05T01:13:16.054460+0000","last_fullsized":"2024-09-05T01:13:16.054460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.747417+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:13:16.747417+0000","last_peered":"2024-09-05T01:13:16.747417+0000","last_clean":"2024-09-05T01:13:16.747417+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:13:16.747417+0000","last_undegraded":"2024-09-05T01:13:16.747417+0000","last_fullsized":"2024-09-05T01:13:16.747417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055764+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:13:16.055764+0000","last_peered":"2024-09-05T01:13:16.055764+0000","last_clean":"2024-09-05T01:13:16.055764+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:13:16.055764+0000","last_undegraded":"2024-09-05T01:13:16.055764+0000","last_fullsized":"2024-09-05T01:13:16.055764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.855521+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:13:47.855521+0000","last_peered":"2024-09-05T01:13:47.855521+0000","last_clean":"2024-09-05T01:13:47.855521+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:13:47.855521+0000","last_undegraded":"2024-09-05T01:13:47.855521+0000","last_fullsized":"2024-09-05T01:13:47.855521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054307+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:13:16.054307+0000","last_peered":"2024-09-05T01:13:16.054307+0000","last_clean":"2024-09-05T01:13:16.054307+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:13:16.054307+0000","last_undegraded":"2024-09-05T01:13:16.054307+0000","last_fullsized":"2024-09-05T01:13:16.054307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055298+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:13:16.055298+0000","last_peered":"2024-09-05T01:13:16.055298+0000","last_clean":"2024-09-05T01:13:16.055298+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:13:16.055298+0000","last_undegraded":"2024-09-05T01:13:16.055298+0000","last_fullsized":"2024-09-05T01:13:16.055298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912617+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:13:15.912617+0000","last_peered":"2024-09-05T01:13:15.912617+0000","last_clean":"2024-09-05T01:13:15.912617+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:13:15.912617+0000","last_undegraded":"2024-09-05T01:13:15.912617+0000","last_fullsized":"2024-09-05T01:13:15.912617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054950+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:13:16.054950+0000","last_peered":"2024-09-05T01:13:16.054950+0000","last_clean":"2024-09-05T01:13:16.054950+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:13:16.054950+0000","last_undegraded":"2024-09-05T01:13:16.054950+0000","last_fullsized":"2024-09-05T01:13:16.054950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055726+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:13:16.055726+0000","last_peered":"2024-09-05T01:13:16.055726+0000","last_clean":"2024-09-05T01:13:16.055726+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:13:16.055726+0000","last_undegraded":"2024-09-05T01:13:16.055726+0000","last_fullsized":"2024-09-05T01:13:16.055726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055186+0000","last_change":"2024-09-05T01:13:03.877073+0000","last_active":"2024-09-05T01:13:16.055186+0000","last_peered":"2024-09-05T01:13:16.055186+0000","last_clean":"2024-09-05T01:13:16.055186+0000","last_became_active":"2024-09-05T01:13:03.876801+0000","last_became_peered":"2024-09-05T01:13:03.876801+0000","last_unstale":"2024-09-05T01:13:16.055186+0000","last_undegraded":"2024-09-05T01:13:16.055186+0000","last_fullsized":"2024-09-05T01:13:16.055186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:13:24.959411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911632+0000","last_change":"2024-09-05T01:13:12.926799+0000","last_active":"2024-09-05T01:13:15.911632+0000","last_peered":"2024-09-05T01:13:15.911632+0000","last_clean":"2024-09-05T01:13:15.911632+0000","last_became_active":"2024-09-05T01:13:12.925704+0000","last_became_peered":"2024-09-05T01:13:12.925704+0000","last_unstale":"2024-09-05T01:13:15.911632+0000","last_undegraded":"2024-09-05T01:13:15.911632+0000","last_fullsized":"2024-09-05T01:13:15.911632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:25.920594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425056+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:13:16.425056+0000","last_peered":"2024-09-05T01:13:16.425056+0000","last_clean":"2024-09-05T01:13:16.425056+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:13:16.425056+0000","last_undegraded":"2024-09-05T01:13:16.425056+0000","last_fullsized":"2024-09-05T01:13:16.425056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:48.850817+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:13:48.850817+0000","last_peered":"2024-09-05T01:13:48.850817+0000","last_clean":"2024-09-05T01:13:48.850817+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:13:48.850817+0000","last_undegraded":"2024-09-05T01:13:48.850817+0000","last_fullsized":"2024-09-05T01:13:48.850817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911788+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:13:15.911788+0000","last_peered":"2024-09-05T01:13:15.911788+0000","last_clean":"2024-09-05T01:13:15.911788+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:13:15.911788+0000","last_undegraded":"2024-09-05T01:13:15.911788+0000","last_fullsized":"2024-09-05T01:13:15.911788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911806+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:13:15.911806+0000","last_peered":"2024-09-05T01:13:15.911806+0000","last_clean":"2024-09-05T01:13:15.911806+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:13:15.911806+0000","last_undegraded":"2024-09-05T01:13:15.911806+0000","last_fullsized":"2024-09-05T01:13:15.911806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.856087+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:13:47.856087+0000","last_peered":"2024-09-05T01:13:47.856087+0000","last_clean":"2024-09-05T01:13:47.856087+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:13:47.856087+0000","last_undegraded":"2024-09-05T01:13:47.856087+0000","last_fullsized":"2024-09-05T01:13:47.856087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055095+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:13:16.055095+0000","last_peered":"2024-09-05T01:13:16.055095+0000","last_clean":"2024-09-05T01:13:16.055095+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:13:16.055095+0000","last_undegraded":"2024-09-05T01:13:16.055095+0000","last_fullsized":"2024-09-05T01:13:16.055095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054223+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:13:16.054223+0000","last_peered":"2024-09-05T01:13:16.054223+0000","last_clean":"2024-09-05T01:13:16.054223+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:13:16.054223+0000","last_undegraded":"2024-09-05T01:13:16.054223+0000","last_fullsized":"2024-09-05T01:13:16.054223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:59.873379+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:13:59.873379+0000","last_peered":"2024-09-05T01:13:59.873379+0000","last_clean":"2024-09-05T01:13:59.873379+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:13:59.873379+0000","last_undegraded":"2024-09-05T01:13:59.873379+0000","last_fullsized":"2024-09-05T01:13:59.873379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055637+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:13:16.055637+0000","last_peered":"2024-09-05T01:13:16.055637+0000","last_clean":"2024-09-05T01:13:16.055637+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:13:16.055637+0000","last_undegraded":"2024-09-05T01:13:16.055637+0000","last_fullsized":"2024-09-05T01:13:16.055637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055249+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:13:16.055249+0000","last_peered":"2024-09-05T01:13:16.055249+0000","last_clean":"2024-09-05T01:13:16.055249+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:13:16.055249+0000","last_undegraded":"2024-09-05T01:13:16.055249+0000","last_fullsized":"2024-09-05T01:13:16.055249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845243+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:13:46.845243+0000","last_peered":"2024-09-05T01:13:46.845243+0000","last_clean":"2024-09-05T01:13:46.845243+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:13:46.845243+0000","last_undegraded":"2024-09-05T01:13:46.845243+0000","last_fullsized":"2024-09-05T01:13:46.845243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424763+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:13:16.424763+0000","last_peered":"2024-09-05T01:13:16.424763+0000","last_clean":"2024-09-05T01:13:16.424763+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:13:16.424763+0000","last_undegraded":"2024-09-05T01:13:16.424763+0000","last_fullsized":"2024-09-05T01:13:16.424763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054509+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:13:16.054509+0000","last_peered":"2024-09-05T01:13:16.054509+0000","last_clean":"2024-09-05T01:13:16.054509+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:13:16.054509+0000","last_undegraded":"2024-09-05T01:13:16.054509+0000","last_fullsized":"2024-09-05T01:13:16.054509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854821+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:13:47.854821+0000","last_peered":"2024-09-05T01:13:47.854821+0000","last_clean":"2024-09-05T01:13:47.854821+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:13:47.854821+0000","last_undegraded":"2024-09-05T01:13:47.854821+0000","last_fullsized":"2024-09-05T01:13:47.854821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912464+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:13:15.912464+0000","last_peered":"2024-09-05T01:13:15.912464+0000","last_clean":"2024-09-05T01:13:15.912464+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:13:15.912464+0000","last_undegraded":"2024-09-05T01:13:15.912464+0000","last_fullsized":"2024-09-05T01:13:15.912464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054881+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:13:16.054881+0000","last_peered":"2024-09-05T01:13:16.054881+0000","last_clean":"2024-09-05T01:13:16.054881+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:13:16.054881+0000","last_undegraded":"2024-09-05T01:13:16.054881+0000","last_fullsized":"2024-09-05T01:13:16.054881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:01.878127+0000","last_change":"2024-09-05T01:13:03.883605+0000","last_active":"2024-09-05T01:14:01.878127+0000","last_peered":"2024-09-05T01:14:01.878127+0000","last_clean":"2024-09-05T01:14:01.878127+0000","last_became_active":"2024-09-05T01:13:03.883275+0000","last_became_peered":"2024-09-05T01:13:03.883275+0000","last_unstale":"2024-09-05T01:14:01.878127+0000","last_undegraded":"2024-09-05T01:14:01.878127+0000","last_fullsized":"2024-09-05T01:14:01.878127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:58:39.763930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054970+0000","last_change":"2024-09-05T01:13:12.917629+0000","last_active":"2024-09-05T01:13:16.054970+0000","last_peered":"2024-09-05T01:13:16.054970+0000","last_clean":"2024-09-05T01:13:16.054970+0000","last_became_active":"2024-09-05T01:13:12.917128+0000","last_became_peered":"2024-09-05T01:13:12.917128+0000","last_unstale":"2024-09-05T01:13:16.054970+0000","last_undegraded":"2024-09-05T01:13:16.054970+0000","last_fullsized":"2024-09-05T01:13:16.054970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:25:12.205059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055092+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:13:16.055092+0000","last_peered":"2024-09-05T01:13:16.055092+0000","last_clean":"2024-09-05T01:13:16.055092+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:13:16.055092+0000","last_undegraded":"2024-09-05T01:13:16.055092+0000","last_fullsized":"2024-09-05T01:13:16.055092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845420+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:13:46.845420+0000","last_peered":"2024-09-05T01:13:46.845420+0000","last_clean":"2024-09-05T01:13:46.845420+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:13:46.845420+0000","last_undegraded":"2024-09-05T01:13:46.845420+0000","last_fullsized":"2024-09-05T01:13:46.845420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912852+0000","last_change":"2024-09-05T01:13:12.923283+0000","last_active":"2024-09-05T01:13:15.912852+0000","last_peered":"2024-09-05T01:13:15.912852+0000","last_clean":"2024-09-05T01:13:15.912852+0000","last_became_active":"2024-09-05T01:13:12.922887+0000","last_became_peered":"2024-09-05T01:13:12.922887+0000","last_unstale":"2024-09-05T01:13:15.912852+0000","last_undegraded":"2024-09-05T01:13:15.912852+0000","last_fullsized":"2024-09-05T01:13:15.912852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:13:39.171952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054760+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:13:16.054760+0000","last_peered":"2024-09-05T01:13:16.054760+0000","last_clean":"2024-09-05T01:13:16.054760+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:13:16.054760+0000","last_undegraded":"2024-09-05T01:13:16.054760+0000","last_fullsized":"2024-09-05T01:13:16.054760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.844469+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:13:46.844469+0000","last_peered":"2024-09-05T01:13:46.844469+0000","last_clean":"2024-09-05T01:13:46.844469+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:13:46.844469+0000","last_undegraded":"2024-09-05T01:13:46.844469+0000","last_fullsized":"2024-09-05T01:13:46.844469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054205+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:13:16.054205+0000","last_peered":"2024-09-05T01:13:16.054205+0000","last_clean":"2024-09-05T01:13:16.054205+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:13:16.054205+0000","last_undegraded":"2024-09-05T01:13:16.054205+0000","last_fullsized":"2024-09-05T01:13:16.054205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055372+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:13:16.055372+0000","last_peered":"2024-09-05T01:13:16.055372+0000","last_clean":"2024-09-05T01:13:16.055372+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:13:16.055372+0000","last_undegraded":"2024-09-05T01:13:16.055372+0000","last_fullsized":"2024-09-05T01:13:16.055372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:00.874680+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:14:00.874680+0000","last_peered":"2024-09-05T01:14:00.874680+0000","last_clean":"2024-09-05T01:14:00.874680+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:14:00.874680+0000","last_undegraded":"2024-09-05T01:14:00.874680+0000","last_fullsized":"2024-09-05T01:14:00.874680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911128+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:13:15.911128+0000","last_peered":"2024-09-05T01:13:15.911128+0000","last_clean":"2024-09-05T01:13:15.911128+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:13:15.911128+0000","last_undegraded":"2024-09-05T01:13:15.911128+0000","last_fullsized":"2024-09-05T01:13:15.911128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054359+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:13:16.054359+0000","last_peered":"2024-09-05T01:13:16.054359+0000","last_clean":"2024-09-05T01:13:16.054359+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:13:16.054359+0000","last_undegraded":"2024-09-05T01:13:16.054359+0000","last_fullsized":"2024-09-05T01:13:16.054359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054578+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:13:16.054578+0000","last_peered":"2024-09-05T01:13:16.054578+0000","last_clean":"2024-09-05T01:13:16.054578+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:13:16.054578+0000","last_undegraded":"2024-09-05T01:13:16.054578+0000","last_fullsized":"2024-09-05T01:13:16.054578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911957+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:13:15.911957+0000","last_peered":"2024-09-05T01:13:15.911957+0000","last_clean":"2024-09-05T01:13:15.911957+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:13:15.911957+0000","last_undegraded":"2024-09-05T01:13:15.911957+0000","last_fullsized":"2024-09-05T01:13:15.911957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424902+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:13:16.424902+0000","last_peered":"2024-09-05T01:13:16.424902+0000","last_clean":"2024-09-05T01:13:16.424902+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:13:16.424902+0000","last_undegraded":"2024-09-05T01:13:16.424902+0000","last_fullsized":"2024-09-05T01:13:16.424902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911111+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:13:15.911111+0000","last_peered":"2024-09-05T01:13:15.911111+0000","last_clean":"2024-09-05T01:13:15.911111+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:13:15.911111+0000","last_undegraded":"2024-09-05T01:13:15.911111+0000","last_fullsized":"2024-09-05T01:13:15.911111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.853721+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:13:47.853721+0000","last_peered":"2024-09-05T01:13:47.853721+0000","last_clean":"2024-09-05T01:13:47.853721+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:13:47.853721+0000","last_undegraded":"2024-09-05T01:13:47.853721+0000","last_fullsized":"2024-09-05T01:13:47.853721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854008+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:13:47.854008+0000","last_peered":"2024-09-05T01:13:47.854008+0000","last_clean":"2024-09-05T01:13:47.854008+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:13:47.854008+0000","last_undegraded":"2024-09-05T01:13:47.854008+0000","last_fullsized":"2024-09-05T01:13:47.854008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912066+0000","last_change":"2024-09-05T01:13:05.899375+0000","last_active":"2024-09-05T01:13:15.912066+0000","last_peered":"2024-09-05T01:13:15.912066+0000","last_clean":"2024-09-05T01:13:15.912066+0000","last_became_active":"2024-09-05T01:13:05.899058+0000","last_became_peered":"2024-09-05T01:13:05.899058+0000","last_unstale":"2024-09-05T01:13:15.912066+0000","last_undegraded":"2024-09-05T01:13:15.912066+0000","last_fullsized":"2024-09-05T01:13:15.912066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:48:55.821585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054408+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:13:16.054408+0000","last_peered":"2024-09-05T01:13:16.054408+0000","last_clean":"2024-09-05T01:13:16.054408+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:13:16.054408+0000","last_undegraded":"2024-09-05T01:13:16.054408+0000","last_fullsized":"2024-09-05T01:13:16.054408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:00.875157+0000","last_change":"2024-09-05T01:13:03.885433+0000","last_active":"2024-09-05T01:14:00.875157+0000","last_peered":"2024-09-05T01:14:00.875157+0000","last_clean":"2024-09-05T01:14:00.875157+0000","last_became_active":"2024-09-05T01:13:03.885051+0000","last_became_peered":"2024-09-05T01:13:03.885051+0000","last_unstale":"2024-09-05T01:14:00.875157+0000","last_undegraded":"2024-09-05T01:14:00.875157+0000","last_fullsized":"2024-09-05T01:14:00.875157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:26:56.989141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054606+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:13:16.054606+0000","last_peered":"2024-09-05T01:13:16.054606+0000","last_clean":"2024-09-05T01:13:16.054606+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:13:16.054606+0000","last_undegraded":"2024-09-05T01:13:16.054606+0000","last_fullsized":"2024-09-05T01:13:16.054606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054965+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:13:16.054965+0000","last_peered":"2024-09-05T01:13:16.054965+0000","last_clean":"2024-09-05T01:13:16.054965+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:13:16.054965+0000","last_undegraded":"2024-09-05T01:13:16.054965+0000","last_fullsized":"2024-09-05T01:13:16.054965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424588+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:13:16.424588+0000","last_peered":"2024-09-05T01:13:16.424588+0000","last_clean":"2024-09-05T01:13:16.424588+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:13:16.424588+0000","last_undegraded":"2024-09-05T01:13:16.424588+0000","last_fullsized":"2024-09-05T01:13:16.424588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054499+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:13:16.054499+0000","last_peered":"2024-09-05T01:13:16.054499+0000","last_clean":"2024-09-05T01:13:16.054499+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:13:16.054499+0000","last_undegraded":"2024-09-05T01:13:16.054499+0000","last_fullsized":"2024-09-05T01:13:16.054499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054163+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:13:16.054163+0000","last_peered":"2024-09-05T01:13:16.054163+0000","last_clean":"2024-09-05T01:13:16.054163+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:13:16.054163+0000","last_undegraded":"2024-09-05T01:13:16.054163+0000","last_fullsized":"2024-09-05T01:13:16.054163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055040+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:13:16.055040+0000","last_peered":"2024-09-05T01:13:16.055040+0000","last_clean":"2024-09-05T01:13:16.055040+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:13:16.055040+0000","last_undegraded":"2024-09-05T01:13:16.055040+0000","last_fullsized":"2024-09-05T01:13:16.055040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054737+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:13:16.054737+0000","last_peered":"2024-09-05T01:13:16.054737+0000","last_clean":"2024-09-05T01:13:16.054737+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:13:16.054737+0000","last_undegraded":"2024-09-05T01:13:16.054737+0000","last_fullsized":"2024-09-05T01:13:16.054737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425082+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:13:16.425082+0000","last_peered":"2024-09-05T01:13:16.425082+0000","last_clean":"2024-09-05T01:13:16.425082+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:13:16.425082+0000","last_undegraded":"2024-09-05T01:13:16.425082+0000","last_fullsized":"2024-09-05T01:13:16.425082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054744+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:13:16.054744+0000","last_peered":"2024-09-05T01:13:16.054744+0000","last_clean":"2024-09-05T01:13:16.054744+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:13:16.054744+0000","last_undegraded":"2024-09-05T01:13:16.054744+0000","last_fullsized":"2024-09-05T01:13:16.054744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911749+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:13:15.911749+0000","last_peered":"2024-09-05T01:13:15.911749+0000","last_clean":"2024-09-05T01:13:15.911749+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:13:15.911749+0000","last_undegraded":"2024-09-05T01:13:15.911749+0000","last_fullsized":"2024-09-05T01:13:15.911749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911680+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:13:15.911680+0000","last_peered":"2024-09-05T01:13:15.911680+0000","last_clean":"2024-09-05T01:13:15.911680+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:13:15.911680+0000","last_undegraded":"2024-09-05T01:13:15.911680+0000","last_fullsized":"2024-09-05T01:13:15.911680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911155+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:13:15.911155+0000","last_peered":"2024-09-05T01:13:15.911155+0000","last_clean":"2024-09-05T01:13:15.911155+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:13:15.911155+0000","last_undegraded":"2024-09-05T01:13:15.911155+0000","last_fullsized":"2024-09-05T01:13:15.911155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054909+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:13:16.054909+0000","last_peered":"2024-09-05T01:13:16.054909+0000","last_clean":"2024-09-05T01:13:16.054909+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:13:16.054909+0000","last_undegraded":"2024-09-05T01:13:16.054909+0000","last_fullsized":"2024-09-05T01:13:16.054909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054306+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:13:16.054306+0000","last_peered":"2024-09-05T01:13:16.054306+0000","last_clean":"2024-09-05T01:13:16.054306+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:13:16.054306+0000","last_undegraded":"2024-09-05T01:13:16.054306+0000","last_fullsized":"2024-09-05T01:13:16.054306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424684+0000","last_change":"2024-09-05T01:13:05.898848+0000","last_active":"2024-09-05T01:13:16.424684+0000","last_peered":"2024-09-05T01:13:16.424684+0000","last_clean":"2024-09-05T01:13:16.424684+0000","last_became_active":"2024-09-05T01:13:05.898390+0000","last_became_peered":"2024-09-05T01:13:05.898390+0000","last_unstale":"2024-09-05T01:13:16.424684+0000","last_undegraded":"2024-09-05T01:13:16.424684+0000","last_fullsized":"2024-09-05T01:13:16.424684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:36:57.114979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055125+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:13:16.055125+0000","last_peered":"2024-09-05T01:13:16.055125+0000","last_clean":"2024-09-05T01:13:16.055125+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:13:16.055125+0000","last_undegraded":"2024-09-05T01:13:16.055125+0000","last_fullsized":"2024-09-05T01:13:16.055125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055424+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:13:16.055424+0000","last_peered":"2024-09-05T01:13:16.055424+0000","last_clean":"2024-09-05T01:13:16.055424+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:13:16.055424+0000","last_undegraded":"2024-09-05T01:13:16.055424+0000","last_fullsized":"2024-09-05T01:13:16.055424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425112+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:13:16.425112+0000","last_peered":"2024-09-05T01:13:16.425112+0000","last_clean":"2024-09-05T01:13:16.425112+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:13:16.425112+0000","last_undegraded":"2024-09-05T01:13:16.425112+0000","last_fullsized":"2024-09-05T01:13:16.425112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054349+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:13:16.054349+0000","last_peered":"2024-09-05T01:13:16.054349+0000","last_clean":"2024-09-05T01:13:16.054349+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:13:16.054349+0000","last_undegraded":"2024-09-05T01:13:16.054349+0000","last_fullsized":"2024-09-05T01:13:16.054349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054900+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:13:16.054900+0000","last_peered":"2024-09-05T01:13:16.054900+0000","last_clean":"2024-09-05T01:13:16.054900+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:13:16.054900+0000","last_undegraded":"2024-09-05T01:13:16.054900+0000","last_fullsized":"2024-09-05T01:13:16.054900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054208+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:13:16.054208+0000","last_peered":"2024-09-05T01:13:16.054208+0000","last_clean":"2024-09-05T01:13:16.054208+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:13:16.054208+0000","last_undegraded":"2024-09-05T01:13:16.054208+0000","last_fullsized":"2024-09-05T01:13:16.054208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054220+0000","last_change":"2024-09-05T01:13:05.890006+0000","last_active":"2024-09-05T01:13:16.054220+0000","last_peered":"2024-09-05T01:13:16.054220+0000","last_clean":"2024-09-05T01:13:16.054220+0000","last_became_active":"2024-09-05T01:13:05.889677+0000","last_became_peered":"2024-09-05T01:13:05.889677+0000","last_unstale":"2024-09-05T01:13:16.054220+0000","last_undegraded":"2024-09-05T01:13:16.054220+0000","last_fullsized":"2024-09-05T01:13:16.054220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:04.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913350+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:13:15.913350+0000","last_peered":"2024-09-05T01:13:15.913350+0000","last_clean":"2024-09-05T01:13:15.913350+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:13:15.913350+0000","last_undegraded":"2024-09-05T01:13:15.913350+0000","last_fullsized":"2024-09-05T01:13:15.913350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055737+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:13:16.055737+0000","last_peered":"2024-09-05T01:13:16.055737+0000","last_clean":"2024-09-05T01:13:16.055737+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:13:16.055737+0000","last_undegraded":"2024-09-05T01:13:16.055737+0000","last_fullsized":"2024-09-05T01:13:16.055737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.056257+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:13:16.056257+0000","last_peered":"2024-09-05T01:13:16.056257+0000","last_clean":"2024-09-05T01:13:16.056257+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:13:16.056257+0000","last_undegraded":"2024-09-05T01:13:16.056257+0000","last_fullsized":"2024-09-05T01:13:16.056257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055264+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:13:16.055264+0000","last_peered":"2024-09-05T01:13:16.055264+0000","last_clean":"2024-09-05T01:13:16.055264+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:13:16.055264+0000","last_undegraded":"2024-09-05T01:13:16.055264+0000","last_fullsized":"2024-09-05T01:13:16.055264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911105+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:13:15.911105+0000","last_peered":"2024-09-05T01:13:15.911105+0000","last_clean":"2024-09-05T01:13:15.911105+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:13:15.911105+0000","last_undegraded":"2024-09-05T01:13:15.911105+0000","last_fullsized":"2024-09-05T01:13:15.911105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054798+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:13:16.054798+0000","last_peered":"2024-09-05T01:13:16.054798+0000","last_clean":"2024-09-05T01:13:16.054798+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:13:16.054798+0000","last_undegraded":"2024-09-05T01:13:16.054798+0000","last_fullsized":"2024-09-05T01:13:16.054798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055427+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:13:16.055427+0000","last_peered":"2024-09-05T01:13:16.055427+0000","last_clean":"2024-09-05T01:13:16.055427+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:13:16.055427+0000","last_undegraded":"2024-09-05T01:13:16.055427+0000","last_fullsized":"2024-09-05T01:13:16.055427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054404+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:13:16.054404+0000","last_peered":"2024-09-05T01:13:16.054404+0000","last_clean":"2024-09-05T01:13:16.054404+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:13:16.054404+0000","last_undegraded":"2024-09-05T01:13:16.054404+0000","last_fullsized":"2024-09-05T01:13:16.054404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913196+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:13:15.913196+0000","last_peered":"2024-09-05T01:13:15.913196+0000","last_clean":"2024-09-05T01:13:15.913196+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:13:15.913196+0000","last_undegraded":"2024-09-05T01:13:15.913196+0000","last_fullsized":"2024-09-05T01:13:15.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054861+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:13:16.054861+0000","last_peered":"2024-09-05T01:13:16.054861+0000","last_clean":"2024-09-05T01:13:16.054861+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:13:16.054861+0000","last_undegraded":"2024-09-05T01:13:16.054861+0000","last_fullsized":"2024-09-05T01:13:16.054861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054459+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:13:16.054459+0000","last_peered":"2024-09-05T01:13:16.054459+0000","last_clean":"2024-09-05T01:13:16.054459+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:13:16.054459+0000","last_undegraded":"2024-09-05T01:13:16.054459+0000","last_fullsized":"2024-09-05T01:13:16.054459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425113+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:13:16.425113+0000","last_peered":"2024-09-05T01:13:16.425113+0000","last_clean":"2024-09-05T01:13:16.425113+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:13:16.425113+0000","last_undegraded":"2024-09-05T01:13:16.425113+0000","last_fullsized":"2024-09-05T01:13:16.425113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56411,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":116,"num_write":155,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":160170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":476607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1116283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972385,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":480939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":476763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1072674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":472019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1068262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":476669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":472213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":526080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":515274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":473407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:14:05.665 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:14:05.911 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 -- 172.21.15.129:0/156260441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c102790 msgr2=0x7ffb8c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 --2- 172.21.15.129:0/156260441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c102790 0x7ffb8c1067e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009900 tx=0x7ffb7c02a410 comp rx=0 tx=0).stop 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 -- 172.21.15.129:0/156260441 shutdown_connections 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 --2- 172.21.15.129:0/156260441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c106f10 0x7ffb8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 --2- 172.21.15.129:0/156260441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c102790 0x7ffb8c1067e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 --2- 172.21.15.129:0/156260441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 -- 172.21.15.129:0/156260441 >> 172.21.15.129:0/156260441 conn(0x7ffb8c0fd8c0 msgr2=0x7ffb8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 -- 172.21.15.129:0/156260441 shutdown_connections 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.405+0000 7ffb911c7640 1 -- 172.21.15.129:0/156260441 wait complete. 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.406+0000 7ffb911c7640 1 Processor -- start 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.406+0000 7ffb911c7640 1 -- start start 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c102790 0x7ffb8c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111de0 con 0x7ffb8c106f10 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111c60 con 0x7ffb8c102790 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c111f60 con 0x7ffb8c101de0 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8a59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c102790 0x7ffb8c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:06.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33118/0 (socket says 172.21.15.129:33118) 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 -- 172.21.15.129:0/2731113524 learned_addr learned my addr 172.21.15.129:0/2731113524 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8ad9d640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 -- 172.21.15.129:0/2731113524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 msgr2=0x7ffb8c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 -- 172.21.15.129:0/2731113524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c102790 msgr2=0x7ffb8c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c102790 0x7ffb8c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 -- 172.21.15.129:0/2731113524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c19d370 con 0x7ffb8c106f10 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb8b59e640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400cad0 tx=0x7ffb8400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400ae20 con 0x7ffb8c106f10 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c19d660 con 0x7ffb8c106f10 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb84005020 con 0x7ffb8c106f10 2024-09-05T01:14:06.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400f6b0 con 0x7ffb8c106f10 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.407+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c1a4f40 con 0x7ffb8c106f10 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.409+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c1021c0 con 0x7ffb8c106f10 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.409+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffb840051d0 con 0x7ffb8c106f10 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.410+0000 7ffb73fff640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 0x7ffb740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.410+0000 7ffb8ad9d640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 0x7ffb740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.411+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7ffb840cf400 con 0x7ffb8c106f10 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.411+0000 7ffb8ad9d640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 0x7ffb740a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005e10 tx=0x7ffb80005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:06.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.415+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb84014030 con 0x7ffb8c106f10 2024-09-05T01:14:06.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.578+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb8c062630 con 0x7ffb740a2c10 2024-09-05T01:14:06.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.586+0000 7ffb73fff640 1 -- 172.21.15.129:0/2731113524 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340335 (secure 0 0 0) 0x7ffb8c062630 con 0x7ffb740a2c10 2024-09-05T01:14:06.590 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:06.591 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 msgr2=0x7ffb740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 0x7ffb740a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005e10 tx=0x7ffb80005da0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 msgr2=0x7ffb8c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400cad0 tx=0x7ffb8400cf90 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 shutdown_connections 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb740a2c10 0x7ffb740a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb8c106f10 0x7ffb8c19cc90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb8c102790 0x7ffb8c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 --2- 172.21.15.129:0/2731113524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb8c101de0 0x7ffb8c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 >> 172.21.15.129:0/2731113524 conn(0x7ffb8c0fd8c0 msgr2=0x7ffb8c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:06.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 shutdown_connections 2024-09-05T01:14:06.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:06.590+0000 7ffb911c7640 1 -- 172.21.15.129:0/2731113524 wait complete. 2024-09-05T01:14:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:06 smithi183 ceph-mon[34589]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:06 smithi183 ceph-mon[34589]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-05T01:14:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:06 smithi129 ceph-mon[30979]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:06 smithi129 ceph-mon[30979]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-05T01:14:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:06 smithi194 ceph-mon[34941]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:06 smithi194 ceph-mon[34941]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.4 KiB/s wr, 7 op/s 2024-09-05T01:14:07.328 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-05T01:14:05.760885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647013,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":225,"num_write":343,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387596,"kb_used_data":13388,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554228,"statfs":{"total":1151940427776,"available":1151543529472,"internally_reserved":0,"allocated":13709312,"data_stored":7636253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42196,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":81,"num_write":28,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003360"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":40,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.855040+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:13:47.855040+0000","last_peered":"2024-09-05T01:13:47.855040+0000","last_clean":"2024-09-05T01:13:47.855040+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:13:47.855040+0000","last_undegraded":"2024-09-05T01:13:47.855040+0000","last_fullsized":"2024-09-05T01:13:47.855040+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911634+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:13:15.911634+0000","last_peered":"2024-09-05T01:13:15.911634+0000","last_clean":"2024-09-05T01:13:15.911634+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:13:15.911634+0000","last_undegraded":"2024-09-05T01:13:15.911634+0000","last_fullsized":"2024-09-05T01:13:15.911634+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054088+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:13:16.054088+0000","last_peered":"2024-09-05T01:13:16.054088+0000","last_clean":"2024-09-05T01:13:16.054088+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:13:16.054088+0000","last_undegraded":"2024-09-05T01:13:16.054088+0000","last_fullsized":"2024-09-05T01:13:16.054088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911825+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:13:15.911825+0000","last_peered":"2024-09-05T01:13:15.911825+0000","last_clean":"2024-09-05T01:13:15.911825+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:13:15.911825+0000","last_undegraded":"2024-09-05T01:13:15.911825+0000","last_fullsized":"2024-09-05T01:13:15.911825+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054671+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:13:16.054671+0000","last_peered":"2024-09-05T01:13:16.054671+0000","last_clean":"2024-09-05T01:13:16.054671+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:13:16.054671+0000","last_undegraded":"2024-09-05T01:13:16.054671+0000","last_fullsized":"2024-09-05T01:13:16.054671+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912648+0000","last_change":"2024-09-05T01:13:12.925330+0000","last_active":"2024-09-05T01:13:15.912648+0000","last_peered":"2024-09-05T01:13:15.912648+0000","last_clean":"2024-09-05T01:13:15.912648+0000","last_became_active":"2024-09-05T01:13:12.924809+0000","last_became_peered":"2024-09-05T01:13:12.924809+0000","last_unstale":"2024-09-05T01:13:15.912648+0000","last_undegraded":"2024-09-05T01:13:15.912648+0000","last_fullsized":"2024-09-05T01:13:15.912648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:18:10.430813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854328+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:13:47.854328+0000","last_peered":"2024-09-05T01:13:47.854328+0000","last_clean":"2024-09-05T01:13:47.854328+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:13:47.854328+0000","last_undegraded":"2024-09-05T01:13:47.854328+0000","last_fullsized":"2024-09-05T01:13:47.854328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055014+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:13:16.055014+0000","last_peered":"2024-09-05T01:13:16.055014+0000","last_clean":"2024-09-05T01:13:16.055014+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:13:16.055014+0000","last_undegraded":"2024-09-05T01:13:16.055014+0000","last_fullsized":"2024-09-05T01:13:16.055014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913036+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:13:15.913036+0000","last_peered":"2024-09-05T01:13:15.913036+0000","last_clean":"2024-09-05T01:13:15.913036+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:13:15.913036+0000","last_undegraded":"2024-09-05T01:13:15.913036+0000","last_fullsized":"2024-09-05T01:13:15.913036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054769+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:13:16.054769+0000","last_peered":"2024-09-05T01:13:16.054769+0000","last_clean":"2024-09-05T01:13:16.054769+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:13:16.054769+0000","last_undegraded":"2024-09-05T01:13:16.054769+0000","last_fullsized":"2024-09-05T01:13:16.054769+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912337+0000","last_change":"2024-09-05T01:13:05.901886+0000","last_active":"2024-09-05T01:13:15.912337+0000","last_peered":"2024-09-05T01:13:15.912337+0000","last_clean":"2024-09-05T01:13:15.912337+0000","last_became_active":"2024-09-05T01:13:05.901565+0000","last_became_peered":"2024-09-05T01:13:05.901565+0000","last_unstale":"2024-09-05T01:13:15.912337+0000","last_undegraded":"2024-09-05T01:13:15.912337+0000","last_fullsized":"2024-09-05T01:13:15.912337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:50:21.707874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912764+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:13:15.912764+0000","last_peered":"2024-09-05T01:13:15.912764+0000","last_clean":"2024-09-05T01:13:15.912764+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:13:15.912764+0000","last_undegraded":"2024-09-05T01:13:15.912764+0000","last_fullsized":"2024-09-05T01:13:15.912764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054678+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:13:16.054678+0000","last_peered":"2024-09-05T01:13:16.054678+0000","last_clean":"2024-09-05T01:13:16.054678+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:13:16.054678+0000","last_undegraded":"2024-09-05T01:13:16.054678+0000","last_fullsized":"2024-09-05T01:13:16.054678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845103+0000","last_change":"2024-09-05T01:13:03.882357+0000","last_active":"2024-09-05T01:13:46.845103+0000","last_peered":"2024-09-05T01:13:46.845103+0000","last_clean":"2024-09-05T01:13:46.845103+0000","last_became_active":"2024-09-05T01:13:03.881983+0000","last_became_peered":"2024-09-05T01:13:03.881983+0000","last_unstale":"2024-09-05T01:13:46.845103+0000","last_undegraded":"2024-09-05T01:13:46.845103+0000","last_fullsized":"2024-09-05T01:13:46.845103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:09:44.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913310+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:13:15.913310+0000","last_peered":"2024-09-05T01:13:15.913310+0000","last_clean":"2024-09-05T01:13:15.913310+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:13:15.913310+0000","last_undegraded":"2024-09-05T01:13:15.913310+0000","last_fullsized":"2024-09-05T01:13:15.913310+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911262+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:13:15.911262+0000","last_peered":"2024-09-05T01:13:15.911262+0000","last_clean":"2024-09-05T01:13:15.911262+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:13:15.911262+0000","last_undegraded":"2024-09-05T01:13:15.911262+0000","last_fullsized":"2024-09-05T01:13:15.911262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054655+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:13:16.054655+0000","last_peered":"2024-09-05T01:13:16.054655+0000","last_clean":"2024-09-05T01:13:16.054655+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:13:16.054655+0000","last_undegraded":"2024-09-05T01:13:16.054655+0000","last_fullsized":"2024-09-05T01:13:16.054655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.863054+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:13:16.863054+0000","last_peered":"2024-09-05T01:13:16.863054+0000","last_clean":"2024-09-05T01:13:16.863054+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:13:16.863054+0000","last_undegraded":"2024-09-05T01:13:16.863054+0000","last_fullsized":"2024-09-05T01:13:16.863054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424775+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:13:16.424775+0000","last_peered":"2024-09-05T01:13:16.424775+0000","last_clean":"2024-09-05T01:13:16.424775+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:13:16.424775+0000","last_undegraded":"2024-09-05T01:13:16.424775+0000","last_fullsized":"2024-09-05T01:13:16.424775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.843904+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:13:46.843904+0000","last_peered":"2024-09-05T01:13:46.843904+0000","last_clean":"2024-09-05T01:13:46.843904+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:13:46.843904+0000","last_undegraded":"2024-09-05T01:13:46.843904+0000","last_fullsized":"2024-09-05T01:13:46.843904+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055534+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:13:16.055534+0000","last_peered":"2024-09-05T01:13:16.055534+0000","last_clean":"2024-09-05T01:13:16.055534+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:13:16.055534+0000","last_undegraded":"2024-09-05T01:13:16.055534+0000","last_fullsized":"2024-09-05T01:13:16.055534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054171+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:13:16.054171+0000","last_peered":"2024-09-05T01:13:16.054171+0000","last_clean":"2024-09-05T01:13:16.054171+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:13:16.054171+0000","last_undegraded":"2024-09-05T01:13:16.054171+0000","last_fullsized":"2024-09-05T01:13:16.054171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.844682+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:13:46.844682+0000","last_peered":"2024-09-05T01:13:46.844682+0000","last_clean":"2024-09-05T01:13:46.844682+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:13:46.844682+0000","last_undegraded":"2024-09-05T01:13:46.844682+0000","last_fullsized":"2024-09-05T01:13:46.844682+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913148+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:13:15.913148+0000","last_peered":"2024-09-05T01:13:15.913148+0000","last_clean":"2024-09-05T01:13:15.913148+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:13:15.913148+0000","last_undegraded":"2024-09-05T01:13:15.913148+0000","last_fullsized":"2024-09-05T01:13:15.913148+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054551+0000","last_change":"2024-09-05T01:13:05.882797+0000","last_active":"2024-09-05T01:13:16.054551+0000","last_peered":"2024-09-05T01:13:16.054551+0000","last_clean":"2024-09-05T01:13:16.054551+0000","last_became_active":"2024-09-05T01:13:05.882461+0000","last_became_peered":"2024-09-05T01:13:05.882461+0000","last_unstale":"2024-09-05T01:13:16.054551+0000","last_undegraded":"2024-09-05T01:13:16.054551+0000","last_fullsized":"2024-09-05T01:13:16.054551+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:30:00.226190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845406+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:13:46.845406+0000","last_peered":"2024-09-05T01:13:46.845406+0000","last_clean":"2024-09-05T01:13:46.845406+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:13:46.845406+0000","last_undegraded":"2024-09-05T01:13:46.845406+0000","last_fullsized":"2024-09-05T01:13:46.845406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054097+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:13:16.054097+0000","last_peered":"2024-09-05T01:13:16.054097+0000","last_clean":"2024-09-05T01:13:16.054097+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:13:16.054097+0000","last_undegraded":"2024-09-05T01:13:16.054097+0000","last_fullsized":"2024-09-05T01:13:16.054097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911790+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:13:15.911790+0000","last_peered":"2024-09-05T01:13:15.911790+0000","last_clean":"2024-09-05T01:13:15.911790+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:13:15.911790+0000","last_undegraded":"2024-09-05T01:13:15.911790+0000","last_fullsized":"2024-09-05T01:13:15.911790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:59.898777+0000","last_change":"2024-09-05T01:13:03.873898+0000","last_active":"2024-09-05T01:13:59.898777+0000","last_peered":"2024-09-05T01:13:59.898777+0000","last_clean":"2024-09-05T01:13:59.898777+0000","last_became_active":"2024-09-05T01:13:03.873546+0000","last_became_peered":"2024-09-05T01:13:03.873546+0000","last_unstale":"2024-09-05T01:13:59.898777+0000","last_undegraded":"2024-09-05T01:13:59.898777+0000","last_fullsized":"2024-09-05T01:13:59.898777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:45:48.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911014+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:13:15.911014+0000","last_peered":"2024-09-05T01:13:15.911014+0000","last_clean":"2024-09-05T01:13:15.911014+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:13:15.911014+0000","last_undegraded":"2024-09-05T01:13:15.911014+0000","last_fullsized":"2024-09-05T01:13:15.911014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054830+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:13:16.054830+0000","last_peered":"2024-09-05T01:13:16.054830+0000","last_clean":"2024-09-05T01:13:16.054830+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:13:16.054830+0000","last_undegraded":"2024-09-05T01:13:16.054830+0000","last_fullsized":"2024-09-05T01:13:16.054830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.843697+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:13:46.843697+0000","last_peered":"2024-09-05T01:13:46.843697+0000","last_clean":"2024-09-05T01:13:46.843697+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:13:46.843697+0000","last_undegraded":"2024-09-05T01:13:46.843697+0000","last_fullsized":"2024-09-05T01:13:46.843697+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424812+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:13:16.424812+0000","last_peered":"2024-09-05T01:13:16.424812+0000","last_clean":"2024-09-05T01:13:16.424812+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:13:16.424812+0000","last_undegraded":"2024-09-05T01:13:16.424812+0000","last_fullsized":"2024-09-05T01:13:16.424812+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.056018+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:13:16.056018+0000","last_peered":"2024-09-05T01:13:16.056018+0000","last_clean":"2024-09-05T01:13:16.056018+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:13:16.056018+0000","last_undegraded":"2024-09-05T01:13:16.056018+0000","last_fullsized":"2024-09-05T01:13:16.056018+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055406+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:13:16.055406+0000","last_peered":"2024-09-05T01:13:16.055406+0000","last_clean":"2024-09-05T01:13:16.055406+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:13:16.055406+0000","last_undegraded":"2024-09-05T01:13:16.055406+0000","last_fullsized":"2024-09-05T01:13:16.055406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054345+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:13:16.054345+0000","last_peered":"2024-09-05T01:13:16.054345+0000","last_clean":"2024-09-05T01:13:16.054345+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:13:16.054345+0000","last_undegraded":"2024-09-05T01:13:16.054345+0000","last_fullsized":"2024-09-05T01:13:16.054345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055644+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:13:16.055644+0000","last_peered":"2024-09-05T01:13:16.055644+0000","last_clean":"2024-09-05T01:13:16.055644+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:13:16.055644+0000","last_undegraded":"2024-09-05T01:13:16.055644+0000","last_fullsized":"2024-09-05T01:13:16.055644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054834+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:13:16.054834+0000","last_peered":"2024-09-05T01:13:16.054834+0000","last_clean":"2024-09-05T01:13:16.054834+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:13:16.054834+0000","last_undegraded":"2024-09-05T01:13:16.054834+0000","last_fullsized":"2024-09-05T01:13:16.054834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054600+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:13:16.054600+0000","last_peered":"2024-09-05T01:13:16.054600+0000","last_clean":"2024-09-05T01:13:16.054600+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:13:16.054600+0000","last_undegraded":"2024-09-05T01:13:16.054600+0000","last_fullsized":"2024-09-05T01:13:16.054600+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911766+0000","last_change":"2024-09-05T01:13:05.897936+0000","last_active":"2024-09-05T01:13:15.911766+0000","last_peered":"2024-09-05T01:13:15.911766+0000","last_clean":"2024-09-05T01:13:15.911766+0000","last_became_active":"2024-09-05T01:13:05.897610+0000","last_became_peered":"2024-09-05T01:13:05.897610+0000","last_unstale":"2024-09-05T01:13:15.911766+0000","last_undegraded":"2024-09-05T01:13:15.911766+0000","last_fullsized":"2024-09-05T01:13:15.911766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:57:14.754999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:01.875161+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:14:01.875161+0000","last_peered":"2024-09-05T01:14:01.875161+0000","last_clean":"2024-09-05T01:14:01.875161+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:14:01.875161+0000","last_undegraded":"2024-09-05T01:14:01.875161+0000","last_fullsized":"2024-09-05T01:14:01.875161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055254+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:13:16.055254+0000","last_peered":"2024-09-05T01:13:16.055254+0000","last_clean":"2024-09-05T01:13:16.055254+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:13:16.055254+0000","last_undegraded":"2024-09-05T01:13:16.055254+0000","last_fullsized":"2024-09-05T01:13:16.055254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.053984+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:13:16.053984+0000","last_peered":"2024-09-05T01:13:16.053984+0000","last_clean":"2024-09-05T01:13:16.053984+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:13:16.053984+0000","last_undegraded":"2024-09-05T01:13:16.053984+0000","last_fullsized":"2024-09-05T01:13:16.053984+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854613+0000","last_change":"2024-09-05T01:13:03.880884+0000","last_active":"2024-09-05T01:13:47.854613+0000","last_peered":"2024-09-05T01:13:47.854613+0000","last_clean":"2024-09-05T01:13:47.854613+0000","last_became_active":"2024-09-05T01:13:03.880274+0000","last_became_peered":"2024-09-05T01:13:03.880274+0000","last_unstale":"2024-09-05T01:13:47.854613+0000","last_undegraded":"2024-09-05T01:13:47.854613+0000","last_fullsized":"2024-09-05T01:13:47.854613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:24.973934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912322+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:13:15.912322+0000","last_peered":"2024-09-05T01:13:15.912322+0000","last_clean":"2024-09-05T01:13:15.912322+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:13:15.912322+0000","last_undegraded":"2024-09-05T01:13:15.912322+0000","last_fullsized":"2024-09-05T01:13:15.912322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911162+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:13:15.911162+0000","last_peered":"2024-09-05T01:13:15.911162+0000","last_clean":"2024-09-05T01:13:15.911162+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:13:15.911162+0000","last_undegraded":"2024-09-05T01:13:15.911162+0000","last_fullsized":"2024-09-05T01:13:15.911162+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:48.853786+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:13:48.853786+0000","last_peered":"2024-09-05T01:13:48.853786+0000","last_clean":"2024-09-05T01:13:48.853786+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:13:48.853786+0000","last_undegraded":"2024-09-05T01:13:48.853786+0000","last_fullsized":"2024-09-05T01:13:48.853786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054460+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:13:16.054460+0000","last_peered":"2024-09-05T01:13:16.054460+0000","last_clean":"2024-09-05T01:13:16.054460+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:13:16.054460+0000","last_undegraded":"2024-09-05T01:13:16.054460+0000","last_fullsized":"2024-09-05T01:13:16.054460+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.747417+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:13:16.747417+0000","last_peered":"2024-09-05T01:13:16.747417+0000","last_clean":"2024-09-05T01:13:16.747417+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:13:16.747417+0000","last_undegraded":"2024-09-05T01:13:16.747417+0000","last_fullsized":"2024-09-05T01:13:16.747417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055764+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:13:16.055764+0000","last_peered":"2024-09-05T01:13:16.055764+0000","last_clean":"2024-09-05T01:13:16.055764+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:13:16.055764+0000","last_undegraded":"2024-09-05T01:13:16.055764+0000","last_fullsized":"2024-09-05T01:13:16.055764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.855521+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:13:47.855521+0000","last_peered":"2024-09-05T01:13:47.855521+0000","last_clean":"2024-09-05T01:13:47.855521+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:13:47.855521+0000","last_undegraded":"2024-09-05T01:13:47.855521+0000","last_fullsized":"2024-09-05T01:13:47.855521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054307+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:13:16.054307+0000","last_peered":"2024-09-05T01:13:16.054307+0000","last_clean":"2024-09-05T01:13:16.054307+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:13:16.054307+0000","last_undegraded":"2024-09-05T01:13:16.054307+0000","last_fullsized":"2024-09-05T01:13:16.054307+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055298+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:13:16.055298+0000","last_peered":"2024-09-05T01:13:16.055298+0000","last_clean":"2024-09-05T01:13:16.055298+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:13:16.055298+0000","last_undegraded":"2024-09-05T01:13:16.055298+0000","last_fullsized":"2024-09-05T01:13:16.055298+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912617+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:13:15.912617+0000","last_peered":"2024-09-05T01:13:15.912617+0000","last_clean":"2024-09-05T01:13:15.912617+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:13:15.912617+0000","last_undegraded":"2024-09-05T01:13:15.912617+0000","last_fullsized":"2024-09-05T01:13:15.912617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054950+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:13:16.054950+0000","last_peered":"2024-09-05T01:13:16.054950+0000","last_clean":"2024-09-05T01:13:16.054950+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:13:16.054950+0000","last_undegraded":"2024-09-05T01:13:16.054950+0000","last_fullsized":"2024-09-05T01:13:16.054950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055726+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:13:16.055726+0000","last_peered":"2024-09-05T01:13:16.055726+0000","last_clean":"2024-09-05T01:13:16.055726+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:13:16.055726+0000","last_undegraded":"2024-09-05T01:13:16.055726+0000","last_fullsized":"2024-09-05T01:13:16.055726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055186+0000","last_change":"2024-09-05T01:13:03.877073+0000","last_active":"2024-09-05T01:13:16.055186+0000","last_peered":"2024-09-05T01:13:16.055186+0000","last_clean":"2024-09-05T01:13:16.055186+0000","last_became_active":"2024-09-05T01:13:03.876801+0000","last_became_peered":"2024-09-05T01:13:03.876801+0000","last_unstale":"2024-09-05T01:13:16.055186+0000","last_undegraded":"2024-09-05T01:13:16.055186+0000","last_fullsized":"2024-09-05T01:13:16.055186+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:13:24.959411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911632+0000","last_change":"2024-09-05T01:13:12.926799+0000","last_active":"2024-09-05T01:13:15.911632+0000","last_peered":"2024-09-05T01:13:15.911632+0000","last_clean":"2024-09-05T01:13:15.911632+0000","last_became_active":"2024-09-05T01:13:12.925704+0000","last_became_peered":"2024-09-05T01:13:12.925704+0000","last_unstale":"2024-09-05T01:13:15.911632+0000","last_undegraded":"2024-09-05T01:13:15.911632+0000","last_fullsized":"2024-09-05T01:13:15.911632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:49:25.920594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425056+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:13:16.425056+0000","last_peered":"2024-09-05T01:13:16.425056+0000","last_clean":"2024-09-05T01:13:16.425056+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:13:16.425056+0000","last_undegraded":"2024-09-05T01:13:16.425056+0000","last_fullsized":"2024-09-05T01:13:16.425056+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:48.850817+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:13:48.850817+0000","last_peered":"2024-09-05T01:13:48.850817+0000","last_clean":"2024-09-05T01:13:48.850817+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:13:48.850817+0000","last_undegraded":"2024-09-05T01:13:48.850817+0000","last_fullsized":"2024-09-05T01:13:48.850817+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911788+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:13:15.911788+0000","last_peered":"2024-09-05T01:13:15.911788+0000","last_clean":"2024-09-05T01:13:15.911788+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:13:15.911788+0000","last_undegraded":"2024-09-05T01:13:15.911788+0000","last_fullsized":"2024-09-05T01:13:15.911788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911806+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:13:15.911806+0000","last_peered":"2024-09-05T01:13:15.911806+0000","last_clean":"2024-09-05T01:13:15.911806+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:13:15.911806+0000","last_undegraded":"2024-09-05T01:13:15.911806+0000","last_fullsized":"2024-09-05T01:13:15.911806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.856087+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:13:47.856087+0000","last_peered":"2024-09-05T01:13:47.856087+0000","last_clean":"2024-09-05T01:13:47.856087+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:13:47.856087+0000","last_undegraded":"2024-09-05T01:13:47.856087+0000","last_fullsized":"2024-09-05T01:13:47.856087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055095+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:13:16.055095+0000","last_peered":"2024-09-05T01:13:16.055095+0000","last_clean":"2024-09-05T01:13:16.055095+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:13:16.055095+0000","last_undegraded":"2024-09-05T01:13:16.055095+0000","last_fullsized":"2024-09-05T01:13:16.055095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054223+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:13:16.054223+0000","last_peered":"2024-09-05T01:13:16.054223+0000","last_clean":"2024-09-05T01:13:16.054223+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:13:16.054223+0000","last_undegraded":"2024-09-05T01:13:16.054223+0000","last_fullsized":"2024-09-05T01:13:16.054223+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:59.873379+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:13:59.873379+0000","last_peered":"2024-09-05T01:13:59.873379+0000","last_clean":"2024-09-05T01:13:59.873379+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:13:59.873379+0000","last_undegraded":"2024-09-05T01:13:59.873379+0000","last_fullsized":"2024-09-05T01:13:59.873379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055637+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:13:16.055637+0000","last_peered":"2024-09-05T01:13:16.055637+0000","last_clean":"2024-09-05T01:13:16.055637+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:13:16.055637+0000","last_undegraded":"2024-09-05T01:13:16.055637+0000","last_fullsized":"2024-09-05T01:13:16.055637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055249+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:13:16.055249+0000","last_peered":"2024-09-05T01:13:16.055249+0000","last_clean":"2024-09-05T01:13:16.055249+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:13:16.055249+0000","last_undegraded":"2024-09-05T01:13:16.055249+0000","last_fullsized":"2024-09-05T01:13:16.055249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845243+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:13:46.845243+0000","last_peered":"2024-09-05T01:13:46.845243+0000","last_clean":"2024-09-05T01:13:46.845243+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:13:46.845243+0000","last_undegraded":"2024-09-05T01:13:46.845243+0000","last_fullsized":"2024-09-05T01:13:46.845243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424763+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:13:16.424763+0000","last_peered":"2024-09-05T01:13:16.424763+0000","last_clean":"2024-09-05T01:13:16.424763+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:13:16.424763+0000","last_undegraded":"2024-09-05T01:13:16.424763+0000","last_fullsized":"2024-09-05T01:13:16.424763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054509+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:13:16.054509+0000","last_peered":"2024-09-05T01:13:16.054509+0000","last_clean":"2024-09-05T01:13:16.054509+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:13:16.054509+0000","last_undegraded":"2024-09-05T01:13:16.054509+0000","last_fullsized":"2024-09-05T01:13:16.054509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854821+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:13:47.854821+0000","last_peered":"2024-09-05T01:13:47.854821+0000","last_clean":"2024-09-05T01:13:47.854821+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:13:47.854821+0000","last_undegraded":"2024-09-05T01:13:47.854821+0000","last_fullsized":"2024-09-05T01:13:47.854821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912464+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:13:15.912464+0000","last_peered":"2024-09-05T01:13:15.912464+0000","last_clean":"2024-09-05T01:13:15.912464+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:13:15.912464+0000","last_undegraded":"2024-09-05T01:13:15.912464+0000","last_fullsized":"2024-09-05T01:13:15.912464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054881+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:13:16.054881+0000","last_peered":"2024-09-05T01:13:16.054881+0000","last_clean":"2024-09-05T01:13:16.054881+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:13:16.054881+0000","last_undegraded":"2024-09-05T01:13:16.054881+0000","last_fullsized":"2024-09-05T01:13:16.054881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'29","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:01.878127+0000","last_change":"2024-09-05T01:13:03.883605+0000","last_active":"2024-09-05T01:14:01.878127+0000","last_peered":"2024-09-05T01:14:01.878127+0000","last_clean":"2024-09-05T01:14:01.878127+0000","last_became_active":"2024-09-05T01:13:03.883275+0000","last_became_peered":"2024-09-05T01:13:03.883275+0000","last_unstale":"2024-09-05T01:14:01.878127+0000","last_undegraded":"2024-09-05T01:14:01.878127+0000","last_fullsized":"2024-09-05T01:14:01.878127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:58:39.763930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":68,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054970+0000","last_change":"2024-09-05T01:13:12.917629+0000","last_active":"2024-09-05T01:13:16.054970+0000","last_peered":"2024-09-05T01:13:16.054970+0000","last_clean":"2024-09-05T01:13:16.054970+0000","last_became_active":"2024-09-05T01:13:12.917128+0000","last_became_peered":"2024-09-05T01:13:12.917128+0000","last_unstale":"2024-09-05T01:13:16.054970+0000","last_undegraded":"2024-09-05T01:13:16.054970+0000","last_fullsized":"2024-09-05T01:13:16.054970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:25:12.205059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055092+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:13:16.055092+0000","last_peered":"2024-09-05T01:13:16.055092+0000","last_clean":"2024-09-05T01:13:16.055092+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:13:16.055092+0000","last_undegraded":"2024-09-05T01:13:16.055092+0000","last_fullsized":"2024-09-05T01:13:16.055092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.845420+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:13:46.845420+0000","last_peered":"2024-09-05T01:13:46.845420+0000","last_clean":"2024-09-05T01:13:46.845420+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:13:46.845420+0000","last_undegraded":"2024-09-05T01:13:46.845420+0000","last_fullsized":"2024-09-05T01:13:46.845420+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912852+0000","last_change":"2024-09-05T01:13:12.923283+0000","last_active":"2024-09-05T01:13:15.912852+0000","last_peered":"2024-09-05T01:13:15.912852+0000","last_clean":"2024-09-05T01:13:15.912852+0000","last_became_active":"2024-09-05T01:13:12.922887+0000","last_became_peered":"2024-09-05T01:13:12.922887+0000","last_unstale":"2024-09-05T01:13:15.912852+0000","last_undegraded":"2024-09-05T01:13:15.912852+0000","last_fullsized":"2024-09-05T01:13:15.912852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:13:39.171952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054760+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:13:16.054760+0000","last_peered":"2024-09-05T01:13:16.054760+0000","last_clean":"2024-09-05T01:13:16.054760+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:13:16.054760+0000","last_undegraded":"2024-09-05T01:13:16.054760+0000","last_fullsized":"2024-09-05T01:13:16.054760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:46.844469+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:13:46.844469+0000","last_peered":"2024-09-05T01:13:46.844469+0000","last_clean":"2024-09-05T01:13:46.844469+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:13:46.844469+0000","last_undegraded":"2024-09-05T01:13:46.844469+0000","last_fullsized":"2024-09-05T01:13:46.844469+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054205+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:13:16.054205+0000","last_peered":"2024-09-05T01:13:16.054205+0000","last_clean":"2024-09-05T01:13:16.054205+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:13:16.054205+0000","last_undegraded":"2024-09-05T01:13:16.054205+0000","last_fullsized":"2024-09-05T01:13:16.054205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055372+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:13:16.055372+0000","last_peered":"2024-09-05T01:13:16.055372+0000","last_clean":"2024-09-05T01:13:16.055372+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:13:16.055372+0000","last_undegraded":"2024-09-05T01:13:16.055372+0000","last_fullsized":"2024-09-05T01:13:16.055372+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":47,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:00.874680+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:14:00.874680+0000","last_peered":"2024-09-05T01:14:00.874680+0000","last_clean":"2024-09-05T01:14:00.874680+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:14:00.874680+0000","last_undegraded":"2024-09-05T01:14:00.874680+0000","last_fullsized":"2024-09-05T01:14:00.874680+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911128+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:13:15.911128+0000","last_peered":"2024-09-05T01:13:15.911128+0000","last_clean":"2024-09-05T01:13:15.911128+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:13:15.911128+0000","last_undegraded":"2024-09-05T01:13:15.911128+0000","last_fullsized":"2024-09-05T01:13:15.911128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054359+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:13:16.054359+0000","last_peered":"2024-09-05T01:13:16.054359+0000","last_clean":"2024-09-05T01:13:16.054359+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:13:16.054359+0000","last_undegraded":"2024-09-05T01:13:16.054359+0000","last_fullsized":"2024-09-05T01:13:16.054359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054578+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:13:16.054578+0000","last_peered":"2024-09-05T01:13:16.054578+0000","last_clean":"2024-09-05T01:13:16.054578+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:13:16.054578+0000","last_undegraded":"2024-09-05T01:13:16.054578+0000","last_fullsized":"2024-09-05T01:13:16.054578+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911957+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:13:15.911957+0000","last_peered":"2024-09-05T01:13:15.911957+0000","last_clean":"2024-09-05T01:13:15.911957+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:13:15.911957+0000","last_undegraded":"2024-09-05T01:13:15.911957+0000","last_fullsized":"2024-09-05T01:13:15.911957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424902+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:13:16.424902+0000","last_peered":"2024-09-05T01:13:16.424902+0000","last_clean":"2024-09-05T01:13:16.424902+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:13:16.424902+0000","last_undegraded":"2024-09-05T01:13:16.424902+0000","last_fullsized":"2024-09-05T01:13:16.424902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911111+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:13:15.911111+0000","last_peered":"2024-09-05T01:13:15.911111+0000","last_clean":"2024-09-05T01:13:15.911111+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:13:15.911111+0000","last_undegraded":"2024-09-05T01:13:15.911111+0000","last_fullsized":"2024-09-05T01:13:15.911111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.853721+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:13:47.853721+0000","last_peered":"2024-09-05T01:13:47.853721+0000","last_clean":"2024-09-05T01:13:47.853721+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:13:47.853721+0000","last_undegraded":"2024-09-05T01:13:47.853721+0000","last_fullsized":"2024-09-05T01:13:47.853721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:47.854008+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:13:47.854008+0000","last_peered":"2024-09-05T01:13:47.854008+0000","last_clean":"2024-09-05T01:13:47.854008+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:13:47.854008+0000","last_undegraded":"2024-09-05T01:13:47.854008+0000","last_fullsized":"2024-09-05T01:13:47.854008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.912066+0000","last_change":"2024-09-05T01:13:05.899375+0000","last_active":"2024-09-05T01:13:15.912066+0000","last_peered":"2024-09-05T01:13:15.912066+0000","last_clean":"2024-09-05T01:13:15.912066+0000","last_became_active":"2024-09-05T01:13:05.899058+0000","last_became_peered":"2024-09-05T01:13:05.899058+0000","last_unstale":"2024-09-05T01:13:15.912066+0000","last_undegraded":"2024-09-05T01:13:15.912066+0000","last_fullsized":"2024-09-05T01:13:15.912066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:48:55.821585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054408+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:13:16.054408+0000","last_peered":"2024-09-05T01:13:16.054408+0000","last_clean":"2024-09-05T01:13:16.054408+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:13:16.054408+0000","last_undegraded":"2024-09-05T01:13:16.054408+0000","last_fullsized":"2024-09-05T01:13:16.054408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":53,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:14:00.875157+0000","last_change":"2024-09-05T01:13:03.885433+0000","last_active":"2024-09-05T01:14:00.875157+0000","last_peered":"2024-09-05T01:14:00.875157+0000","last_clean":"2024-09-05T01:14:00.875157+0000","last_became_active":"2024-09-05T01:13:03.885051+0000","last_became_peered":"2024-09-05T01:13:03.885051+0000","last_unstale":"2024-09-05T01:14:00.875157+0000","last_undegraded":"2024-09-05T01:14:00.875157+0000","last_fullsized":"2024-09-05T01:14:00.875157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:26:56.989141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054606+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:13:16.054606+0000","last_peered":"2024-09-05T01:13:16.054606+0000","last_clean":"2024-09-05T01:13:16.054606+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:13:16.054606+0000","last_undegraded":"2024-09-05T01:13:16.054606+0000","last_fullsized":"2024-09-05T01:13:16.054606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054965+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:13:16.054965+0000","last_peered":"2024-09-05T01:13:16.054965+0000","last_clean":"2024-09-05T01:13:16.054965+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:13:16.054965+0000","last_undegraded":"2024-09-05T01:13:16.054965+0000","last_fullsized":"2024-09-05T01:13:16.054965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424588+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:13:16.424588+0000","last_peered":"2024-09-05T01:13:16.424588+0000","last_clean":"2024-09-05T01:13:16.424588+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:13:16.424588+0000","last_undegraded":"2024-09-05T01:13:16.424588+0000","last_fullsized":"2024-09-05T01:13:16.424588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054499+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:13:16.054499+0000","last_peered":"2024-09-05T01:13:16.054499+0000","last_clean":"2024-09-05T01:13:16.054499+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:13:16.054499+0000","last_undegraded":"2024-09-05T01:13:16.054499+0000","last_fullsized":"2024-09-05T01:13:16.054499+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054163+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:13:16.054163+0000","last_peered":"2024-09-05T01:13:16.054163+0000","last_clean":"2024-09-05T01:13:16.054163+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:13:16.054163+0000","last_undegraded":"2024-09-05T01:13:16.054163+0000","last_fullsized":"2024-09-05T01:13:16.054163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055040+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:13:16.055040+0000","last_peered":"2024-09-05T01:13:16.055040+0000","last_clean":"2024-09-05T01:13:16.055040+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:13:16.055040+0000","last_undegraded":"2024-09-05T01:13:16.055040+0000","last_fullsized":"2024-09-05T01:13:16.055040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054737+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:13:16.054737+0000","last_peered":"2024-09-05T01:13:16.054737+0000","last_clean":"2024-09-05T01:13:16.054737+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:13:16.054737+0000","last_undegraded":"2024-09-05T01:13:16.054737+0000","last_fullsized":"2024-09-05T01:13:16.054737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425082+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:13:16.425082+0000","last_peered":"2024-09-05T01:13:16.425082+0000","last_clean":"2024-09-05T01:13:16.425082+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:13:16.425082+0000","last_undegraded":"2024-09-05T01:13:16.425082+0000","last_fullsized":"2024-09-05T01:13:16.425082+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054744+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:13:16.054744+0000","last_peered":"2024-09-05T01:13:16.054744+0000","last_clean":"2024-09-05T01:13:16.054744+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:13:16.054744+0000","last_undegraded":"2024-09-05T01:13:16.054744+0000","last_fullsized":"2024-09-05T01:13:16.054744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911749+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:13:15.911749+0000","last_peered":"2024-09-05T01:13:15.911749+0000","last_clean":"2024-09-05T01:13:15.911749+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:13:15.911749+0000","last_undegraded":"2024-09-05T01:13:15.911749+0000","last_fullsized":"2024-09-05T01:13:15.911749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911680+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:13:15.911680+0000","last_peered":"2024-09-05T01:13:15.911680+0000","last_clean":"2024-09-05T01:13:15.911680+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:13:15.911680+0000","last_undegraded":"2024-09-05T01:13:15.911680+0000","last_fullsized":"2024-09-05T01:13:15.911680+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911155+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:13:15.911155+0000","last_peered":"2024-09-05T01:13:15.911155+0000","last_clean":"2024-09-05T01:13:15.911155+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:13:15.911155+0000","last_undegraded":"2024-09-05T01:13:15.911155+0000","last_fullsized":"2024-09-05T01:13:15.911155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054909+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:13:16.054909+0000","last_peered":"2024-09-05T01:13:16.054909+0000","last_clean":"2024-09-05T01:13:16.054909+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:13:16.054909+0000","last_undegraded":"2024-09-05T01:13:16.054909+0000","last_fullsized":"2024-09-05T01:13:16.054909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054306+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:13:16.054306+0000","last_peered":"2024-09-05T01:13:16.054306+0000","last_clean":"2024-09-05T01:13:16.054306+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:13:16.054306+0000","last_undegraded":"2024-09-05T01:13:16.054306+0000","last_fullsized":"2024-09-05T01:13:16.054306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.424684+0000","last_change":"2024-09-05T01:13:05.898848+0000","last_active":"2024-09-05T01:13:16.424684+0000","last_peered":"2024-09-05T01:13:16.424684+0000","last_clean":"2024-09-05T01:13:16.424684+0000","last_became_active":"2024-09-05T01:13:05.898390+0000","last_became_peered":"2024-09-05T01:13:05.898390+0000","last_unstale":"2024-09-05T01:13:16.424684+0000","last_undegraded":"2024-09-05T01:13:16.424684+0000","last_fullsized":"2024-09-05T01:13:16.424684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:36:57.114979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055125+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:13:16.055125+0000","last_peered":"2024-09-05T01:13:16.055125+0000","last_clean":"2024-09-05T01:13:16.055125+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:13:16.055125+0000","last_undegraded":"2024-09-05T01:13:16.055125+0000","last_fullsized":"2024-09-05T01:13:16.055125+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055424+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:13:16.055424+0000","last_peered":"2024-09-05T01:13:16.055424+0000","last_clean":"2024-09-05T01:13:16.055424+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:13:16.055424+0000","last_undegraded":"2024-09-05T01:13:16.055424+0000","last_fullsized":"2024-09-05T01:13:16.055424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425112+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:13:16.425112+0000","last_peered":"2024-09-05T01:13:16.425112+0000","last_clean":"2024-09-05T01:13:16.425112+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:13:16.425112+0000","last_undegraded":"2024-09-05T01:13:16.425112+0000","last_fullsized":"2024-09-05T01:13:16.425112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054349+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:13:16.054349+0000","last_peered":"2024-09-05T01:13:16.054349+0000","last_clean":"2024-09-05T01:13:16.054349+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:13:16.054349+0000","last_undegraded":"2024-09-05T01:13:16.054349+0000","last_fullsized":"2024-09-05T01:13:16.054349+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054900+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:13:16.054900+0000","last_peered":"2024-09-05T01:13:16.054900+0000","last_clean":"2024-09-05T01:13:16.054900+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:13:16.054900+0000","last_undegraded":"2024-09-05T01:13:16.054900+0000","last_fullsized":"2024-09-05T01:13:16.054900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054208+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:13:16.054208+0000","last_peered":"2024-09-05T01:13:16.054208+0000","last_clean":"2024-09-05T01:13:16.054208+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:13:16.054208+0000","last_undegraded":"2024-09-05T01:13:16.054208+0000","last_fullsized":"2024-09-05T01:13:16.054208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054220+0000","last_change":"2024-09-05T01:13:05.890006+0000","last_active":"2024-09-05T01:13:16.054220+0000","last_peered":"2024-09-05T01:13:16.054220+0000","last_clean":"2024-09-05T01:13:16.054220+0000","last_became_active":"2024-09-05T01:13:05.889677+0000","last_became_peered":"2024-09-05T01:13:05.889677+0000","last_unstale":"2024-09-05T01:13:16.054220+0000","last_undegraded":"2024-09-05T01:13:16.054220+0000","last_fullsized":"2024-09-05T01:13:16.054220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:04.109316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913350+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:13:15.913350+0000","last_peered":"2024-09-05T01:13:15.913350+0000","last_clean":"2024-09-05T01:13:15.913350+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:13:15.913350+0000","last_undegraded":"2024-09-05T01:13:15.913350+0000","last_fullsized":"2024-09-05T01:13:15.913350+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055737+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:13:16.055737+0000","last_peered":"2024-09-05T01:13:16.055737+0000","last_clean":"2024-09-05T01:13:16.055737+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:13:16.055737+0000","last_undegraded":"2024-09-05T01:13:16.055737+0000","last_fullsized":"2024-09-05T01:13:16.055737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.056257+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:13:16.056257+0000","last_peered":"2024-09-05T01:13:16.056257+0000","last_clean":"2024-09-05T01:13:16.056257+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:13:16.056257+0000","last_undegraded":"2024-09-05T01:13:16.056257+0000","last_fullsized":"2024-09-05T01:13:16.056257+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055264+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:13:16.055264+0000","last_peered":"2024-09-05T01:13:16.055264+0000","last_clean":"2024-09-05T01:13:16.055264+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:13:16.055264+0000","last_undegraded":"2024-09-05T01:13:16.055264+0000","last_fullsized":"2024-09-05T01:13:16.055264+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.911105+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:13:15.911105+0000","last_peered":"2024-09-05T01:13:15.911105+0000","last_clean":"2024-09-05T01:13:15.911105+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:13:15.911105+0000","last_undegraded":"2024-09-05T01:13:15.911105+0000","last_fullsized":"2024-09-05T01:13:15.911105+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054798+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:13:16.054798+0000","last_peered":"2024-09-05T01:13:16.054798+0000","last_clean":"2024-09-05T01:13:16.054798+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:13:16.054798+0000","last_undegraded":"2024-09-05T01:13:16.054798+0000","last_fullsized":"2024-09-05T01:13:16.054798+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.055427+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:13:16.055427+0000","last_peered":"2024-09-05T01:13:16.055427+0000","last_clean":"2024-09-05T01:13:16.055427+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:13:16.055427+0000","last_undegraded":"2024-09-05T01:13:16.055427+0000","last_fullsized":"2024-09-05T01:13:16.055427+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054404+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:13:16.054404+0000","last_peered":"2024-09-05T01:13:16.054404+0000","last_clean":"2024-09-05T01:13:16.054404+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:13:16.054404+0000","last_undegraded":"2024-09-05T01:13:16.054404+0000","last_fullsized":"2024-09-05T01:13:16.054404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:15.913196+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:13:15.913196+0000","last_peered":"2024-09-05T01:13:15.913196+0000","last_clean":"2024-09-05T01:13:15.913196+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:13:15.913196+0000","last_undegraded":"2024-09-05T01:13:15.913196+0000","last_fullsized":"2024-09-05T01:13:15.913196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054861+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:13:16.054861+0000","last_peered":"2024-09-05T01:13:16.054861+0000","last_clean":"2024-09-05T01:13:16.054861+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:13:16.054861+0000","last_undegraded":"2024-09-05T01:13:16.054861+0000","last_fullsized":"2024-09-05T01:13:16.054861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.054459+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:13:16.054459+0000","last_peered":"2024-09-05T01:13:16.054459+0000","last_clean":"2024-09-05T01:13:16.054459+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:13:16.054459+0000","last_undegraded":"2024-09-05T01:13:16.054459+0000","last_fullsized":"2024-09-05T01:13:16.054459+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-05T01:13:16.425113+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:13:16.425113+0000","last_peered":"2024-09-05T01:13:16.425113+0000","last_clean":"2024-09-05T01:13:16.425113+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:13:16.425113+0000","last_undegraded":"2024-09-05T01:13:16.425113+0000","last_fullsized":"2024-09-05T01:13:16.425113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56411,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":110,"num_read_kb":116,"num_write":160,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":169233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579929,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":966656,"data_stored":476607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1646592,"data_stored":1116283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":480939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":476763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712428,"statfs":{"total":95995035648,"available":95961526272,"internally_reserved":0,"allocated":1576960,"data_stored":1072674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":472019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1585152,"data_stored":1068262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":476669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215168,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":974848,"data_stored":472213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":526080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":524337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":473407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":54151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:14:07.334 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T01:14:07.334 INFO:tasks.cephadm:Setting up client nodes... 2024-09-05T01:14:07.335 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-05T01:14:07.583 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:07.969 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:07 smithi129 ceph-mon[30979]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:08.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 -- 172.21.15.129:0/1544118998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c101c00 msgr2=0x7f701c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 --2- 172.21.15.129:0/1544118998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c101c00 0x7f701c102000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7010009900 tx=0x7f701002a410 comp rx=0 tx=0).stop 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 -- 172.21.15.129:0/1544118998 shutdown_connections 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 --2- 172.21.15.129:0/1544118998 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c1037c0 0x7f701c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 --2- 172.21.15.129:0/1544118998 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 --2- 172.21.15.129:0/1544118998 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c101c00 0x7f701c102000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 -- 172.21.15.129:0/1544118998 >> 172.21.15.129:0/1544118998 conn(0x7f701c0fd390 msgr2=0x7f701c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 -- 172.21.15.129:0/1544118998 shutdown_connections 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.058+0000 7f7022e15640 1 -- 172.21.15.129:0/1544118998 wait complete. 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 Processor -- start 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 -- start start 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 0x7f701c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:08.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10d810 con 0x7f701c1037c0 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10d690 con 0x7f701c102e00 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7022e15640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f701c10d990 con 0x7f701c101c00 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33138/0 (socket says 172.21.15.129:33138) 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 -- 172.21.15.129:0/1577482173 learned_addr learned my addr 172.21.15.129:0/1577482173 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f7020bb1640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 0x7f701c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f701bfff640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 -- 172.21.15.129:0/1577482173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 msgr2=0x7f701c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 0x7f701c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 -- 172.21.15.129:0/1577482173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 msgr2=0x7f701c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:08.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.059+0000 7f70213b2640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f70213b2640 1 -- 172.21.15.129:0/1577482173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0753e0 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f701bfff640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7020bb1640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 0x7f701c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f70213b2640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f701400b970 tx=0x7f701400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7014019070 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7014014440 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a2120 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70140075d0 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.060+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a2660 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.061+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c109140 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.062+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f701401c070 con 0x7f701c1037c0 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.063+0000 7f7019ffb640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 0x7f700c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:08.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.063+0000 7f7020bb1640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 0x7f700c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.063+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f70140ce980 con 0x7f701c1037c0 2024-09-05T01:14:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.063+0000 7f7020bb1640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 0x7f700c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7010006010 tx=0x7f7010000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:08.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.067+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f701409b650 con 0x7f701c1037c0 2024-09-05T01:14:08.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:07 smithi194 ceph-mon[34941]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:08.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.299+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f701c062df0 con 0x7f701c1037c0 2024-09-05T01:14:08.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.301+0000 7f7019ffb640 1 -- 172.21.15.129:0/1577482173 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f701409af50 con 0x7f701c1037c0 2024-09-05T01:14:08.302 INFO:teuthology.orchestra.run.smithi129.stdout:[client.0] 2024-09-05T01:14:08.302 INFO:teuthology.orchestra.run.smithi129.stdout: key = AQDgBdlmIs3lERAAEtGuAss7djxLUNYKeD0fDg== 2024-09-05T01:14:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 msgr2=0x7f700c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 0x7f700c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7010006010 tx=0x7f7010000bc0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 msgr2=0x7f701c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:08.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f701400b970 tx=0x7f701400be30 comp rx=0 tx=0).stop 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 shutdown_connections 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f700c0a2c10 0x7f700c0a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f701c1037c0 0x7f701c074b20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f701c102e00 0x7f701c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 --2- 172.21.15.129:0/1577482173 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f701c101c00 0x7f701c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 >> 172.21.15.129:0/1577482173 conn(0x7f701c0fd390 msgr2=0x7f701c106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 shutdown_connections 2024-09-05T01:14:08.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:08.303+0000 7f7022e15640 1 -- 172.21.15.129:0/1577482173 wait complete. 2024-09-05T01:14:08.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:07 smithi183 ceph-mon[34589]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:08.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:08 smithi129 ceph-mon[30979]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-05T01:14:08.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:08 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T01:14:08.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:08 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T01:14:09.020 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:14:09.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-05T01:14:09.020 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-05T01:14:09.056 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T01:14:09.057 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-05T01:14:09.057 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph mgr dump --format=json 2024-09-05T01:14:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:08 smithi194 ceph-mon[34941]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-05T01:14:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:08 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T01:14:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:08 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T01:14:09.342 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:08 smithi183 ceph-mon[34589]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 7 op/s 2024-09-05T01:14:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:08 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T01:14:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:08 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1577482173' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T01:14:09.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 -- 172.21.15.129:0/3405046249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 msgr2=0x7f65bc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:09.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 --2- 172.21.15.129:0/3405046249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc1006d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009cd0 tx=0x7f65ac02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 -- 172.21.15.129:0/3405046249 shutdown_connections 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 --2- 172.21.15.129:0/3405046249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc106f50 0x7f65bc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 --2- 172.21.15.129:0/3405046249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc100ca0 0x7f65bc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 --2- 172.21.15.129:0/3405046249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc1006d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.816+0000 7f65c33e1640 1 -- 172.21.15.129:0/3405046249 >> 172.21.15.129:0/3405046249 conn(0x7f65bc077210 msgr2=0x7f65bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- 172.21.15.129:0/3405046249 shutdown_connections 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- 172.21.15.129:0/3405046249 wait complete. 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 Processor -- start 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- start start 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 0x7f65bc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc106f50 0x7f65bc19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc111eb0 con 0x7f65bc1002f0 2024-09-05T01:14:09.820 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc111d30 con 0x7f65bc106f50 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.817+0000 7f65c33e1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc112030 con 0x7f65bc100ca0 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33156/0 (socket says 172.21.15.129:33156) 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 -- 172.21.15.129:0/2093687431 learned_addr learned my addr 172.21.15.129:0/2093687431 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c197e640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc106f50 0x7f65bc19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c097c640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 0x7f65bc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 -- 172.21.15.129:0/2093687431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 msgr2=0x7f65bc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 0x7f65bc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 -- 172.21.15.129:0/2093687431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc106f50 msgr2=0x7f65bc19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc106f50 0x7f65bc19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:09.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 -- 172.21.15.129:0/2093687431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc19d3e0 con 0x7f65bc1002f0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c097c640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 0x7f65bc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c117d640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006010 tx=0x7f65ac003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac004450 con 0x7f65bc1002f0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc19d670 con 0x7f65bc1002f0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65ac0045f0 con 0x7f65bc1002f0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0466e0 con 0x7f65bc1002f0 2024-09-05T01:14:09.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.818+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc1a4fb0 con 0x7f65bc1002f0 2024-09-05T01:14:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.819+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65bc1006d0 con 0x7f65bc1002f0 2024-09-05T01:14:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.821+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f65ac046880 con 0x7f65bc1002f0 2024-09-05T01:14:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.821+0000 7f65ba7fc640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 0x7f65a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:09.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.822+0000 7f65c097c640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 0x7f65a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:09.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.822+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f65ac02e030 con 0x7f65bc1002f0 2024-09-05T01:14:09.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.822+0000 7f65c097c640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 0x7f65a40a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0ff860 tx=0x7f65b0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:09.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:09.826+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f65ac04b060 con 0x7f65bc1002f0 2024-09-05T01:14:09.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:09 smithi129 ceph-mon[30979]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-05T01:14:10.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.045+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f65bc10a120 con 0x7f65bc1002f0 2024-09-05T01:14:10.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.049+0000 7f65ba7fc640 1 -- 172.21.15.129:0/2093687431 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263207 (secure 0 0 0) 0x7f65ac0bd740 con 0x7f65bc1002f0 2024-09-05T01:14:10.053 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:10.054 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 msgr2=0x7f65a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 0x7f65a40a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f65bc0ff860 tx=0x7f65b0005ec0 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 msgr2=0x7f65bc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f65ac006010 tx=0x7f65ac003c70 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 shutdown_connections 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65a40a2c10 0x7f65a40a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65bc106f50 0x7f65bc19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65bc100ca0 0x7f65bc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 --2- 172.21.15.129:0/2093687431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65bc1002f0 0x7f65bc198430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 >> 172.21.15.129:0/2093687431 conn(0x7f65bc077210 msgr2=0x7f65bc105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 shutdown_connections 2024-09-05T01:14:10.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:10.052+0000 7f65c33e1640 1 -- 172.21.15.129:0/2093687431 wait complete. 2024-09-05T01:14:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:09 smithi194 ceph-mon[34941]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-05T01:14:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:09 smithi183 ceph-mon[34589]: pgmap v312: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-05T01:14:10.762 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":655841650},{"type":"v1","addr":"172.21.15.129:6801","nonce":655841650}]},"active_addr":"172.21.15.129:6801/655841650","active_change":"2024-09-05T01:06:17.664396+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24128,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.129:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":3866894450}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2101621157}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2624723288}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":2651423209}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.129:0","nonce":1450116612}]}]} 2024-09-05T01:14:10.767 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-05T01:14:10.767 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-05T01:14:10.767 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:14:10.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:10 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2093687431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T01:14:11.013 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:10 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2093687431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T01:14:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:10 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2093687431' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T01:14:11.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 -- 172.21.15.129:0/3875460180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598101dd0 msgr2=0x7f35981021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 --2- 172.21.15.129:0/3875460180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598101dd0 0x7f35981021b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3588009900 tx=0x7f358802a410 comp rx=0 tx=0).stop 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 -- 172.21.15.129:0/3875460180 shutdown_connections 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 --2- 172.21.15.129:0/3875460180 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598106f00 0x7f359810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 --2- 172.21.15.129:0/3875460180 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598102780 0x7f35981067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 --2- 172.21.15.129:0/3875460180 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598101dd0 0x7f35981021b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 -- 172.21.15.129:0/3875460180 >> 172.21.15.129:0/3875460180 conn(0x7f35980fd890 msgr2=0x7f35980ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 -- 172.21.15.129:0/3875460180 shutdown_connections 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.616+0000 7f359d48a640 1 -- 172.21.15.129:0/3875460180 wait complete. 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 Processor -- start 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 -- start start 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598101dd0 0x7f3598198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 0x7f3598198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:11.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3598111eb0 con 0x7f3598106f00 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3598111d30 con 0x7f3598102780 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f359d48a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3598112030 con 0x7f3598101dd0 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f35977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f35977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33166/0 (socket says 172.21.15.129:33166) 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f35977fe640 1 -- 172.21.15.129:0/4022744359 learned_addr learned my addr 172.21.15.129:0/4022744359 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.617+0000 7f3596ffd640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598101dd0 0x7f3598198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35967fc640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 0x7f3598198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 -- 172.21.15.129:0/4022744359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598101dd0 msgr2=0x7f3598198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:11.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598101dd0 0x7f3598198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 -- 172.21.15.129:0/4022744359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 msgr2=0x7f3598198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 0x7f3598198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 -- 172.21.15.129:0/4022744359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359819d540 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35967fc640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 0x7f3598198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f35977fe640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f359000b970 tx=0x7f359000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590019070 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590014440 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359819d830 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900075d0 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.618+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35981a5110 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.619+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3598102230 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.620+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f359001c070 con 0x7f3598106f00 2024-09-05T01:14:11.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.621+0000 7f357ffff640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 0x7f35800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:11.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.621+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f35900ce8d0 con 0x7f3598106f00 2024-09-05T01:14:11.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.621+0000 7f3596ffd640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 0x7f35800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:11.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.621+0000 7f3596ffd640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 0x7f35800a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3588031440 tx=0x7f3588005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:11.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.625+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f359009af50 con 0x7f3598106f00 2024-09-05T01:14:11.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.793+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f359810a110 con 0x7f3598106f00 2024-09-05T01:14:11.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.794+0000 7f357ffff640 1 -- 172.21.15.129:0/4022744359 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7f359009f250 con 0x7f3598106f00 2024-09-05T01:14:11.795 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:11.795 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 msgr2=0x7f35800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 0x7f35800a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3588031440 tx=0x7f3588005e80 comp rx=0 tx=0).stop 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 msgr2=0x7f359819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f359000b970 tx=0x7f359000be30 comp rx=0 tx=0).stop 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 shutdown_connections 2024-09-05T01:14:11.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35800a2c10 0x7f35800a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3598106f00 0x7f359819ce60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3598102780 0x7f3598198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 --2- 172.21.15.129:0/4022744359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3598101dd0 0x7f3598198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 >> 172.21.15.129:0/4022744359 conn(0x7f35980fd890 msgr2=0x7f35980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 shutdown_connections 2024-09-05T01:14:11.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:11.797+0000 7f359d48a640 1 -- 172.21.15.129:0/4022744359 wait complete. 2024-09-05T01:14:12.200 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:11 smithi129 ceph-mon[30979]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-05T01:14:12.200 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:11 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4022744359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:11 smithi194 ceph-mon[34941]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-05T01:14:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:11 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4022744359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:12.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:11 smithi183 ceph-mon[34589]: pgmap v313: 129 pgs: 129 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-05T01:14:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:11 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4022744359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:12.507 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-05T01:14:12.507 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:14:12.753 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:13.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 -- 172.21.15.129:0/785777245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81008c0 msgr2=0x7fddd8100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:13.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 --2- 172.21.15.129:0/785777245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81008c0 0x7fddd8100ca0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a410 comp rx=0 tx=0).stop 2024-09-05T01:14:13.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 -- 172.21.15.129:0/785777245 shutdown_connections 2024-09-05T01:14:13.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 --2- 172.21.15.129:0/785777245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd8105ec0 0x7fddd8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 --2- 172.21.15.129:0/785777245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81011e0 0x7fddd8105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 --2- 172.21.15.129:0/785777245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd81008c0 0x7fddd8100ca0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 -- 172.21.15.129:0/785777245 >> 172.21.15.129:0/785777245 conn(0x7fddd80fc7d0 msgr2=0x7fddd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 -- 172.21.15.129:0/785777245 shutdown_connections 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.245+0000 7fdddf84e640 1 -- 172.21.15.129:0/785777245 wait complete. 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 Processor -- start 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 -- start start 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81008c0 0x7fddd81a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 0x7fddd81a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8118540 con 0x7fddd8105ec0 2024-09-05T01:14:13.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81183c0 con 0x7fddd81008c0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddf84e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81186c0 con 0x7fddd81011e0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddddeb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddddeb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58368/0 (socket says 172.21.15.129:58368) 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddddeb640 1 -- 172.21.15.129:0/948529680 learned_addr learned my addr 172.21.15.129:0/948529680 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddd5ea640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81008c0 0x7fddd81a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.246+0000 7fdddcde9640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 0x7fddd81a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 -- 172.21.15.129:0/948529680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 msgr2=0x7fddd81a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 0x7fddd81a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 -- 172.21.15.129:0/948529680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81008c0 msgr2=0x7fddd81a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81008c0 0x7fddd81a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 -- 172.21.15.129:0/948529680 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd819f080 con 0x7fddd8105ec0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddcde9640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 0x7fddd81a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddddeb640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fddd000b970 tx=0x7fddd000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:13.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd0019070 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd819f370 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fddd0014440 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd00075d0 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.247+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd81ab660 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.248+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd806a850 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.249+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fddd001c070 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.250+0000 7fddce7fc640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.250+0000 7fdddd5ea640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.250+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fddd00ce900 con 0x7fddd8105ec0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.250+0000 7fdddd5ea640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 0x7fddc40a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fddd819fa90 tx=0x7fddd40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:13.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.258+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddd009af80 con 0x7fddd8105ec0 2024-09-05T01:14:13.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.433+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddd81ab8f0 con 0x7fddd8105ec0 2024-09-05T01:14:13.436 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.435+0000 7fddce7fc640 1 -- 172.21.15.129:0/948529680 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24259 (secure 0 0 0) 0x7fddd009f280 con 0x7fddd8105ec0 2024-09-05T01:14:13.437 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:13.437 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":86,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:13:15.572012+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 msgr2=0x7fddc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 0x7fddc40a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fddd819fa90 tx=0x7fddd40058d0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 msgr2=0x7fddd819e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fddd000b970 tx=0x7fddd000be30 comp rx=0 tx=0).stop 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 shutdown_connections 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fddc40a2c10 0x7fddc40a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.439 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddd8105ec0 0x7fddd819e950 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fddd81011e0 0x7fddd81a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 --2- 172.21.15.129:0/948529680 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fddd81008c0 0x7fddd81a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:13.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 >> 172.21.15.129:0/948529680 conn(0x7fddd80fc7d0 msgr2=0x7fddd8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:13.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.437+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 shutdown_connections 2024-09-05T01:14:13.440 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:13.438+0000 7fdddf84e640 1 -- 172.21.15.129:0/948529680 wait complete. 2024-09-05T01:14:13.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:13 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/948529680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:13.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:13 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/948529680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:13.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:13 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/948529680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:14.107 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-05T01:14:14.108 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-05T01:14:14.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-05T01:14:14.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-05T01:14:14.109 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-05T01:14:14.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-05T01:14:14.110 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-05T01:14:14.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-05T01:14:14.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-05T01:14:14.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-05T01:14:14.113 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-05T01:14:14.113 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-05T01:14:14.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:14 smithi194 ceph-mon[34941]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-05T01:14:14.778 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:14.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:14 smithi183 ceph-mon[34589]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-05T01:14:14.917 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:14 smithi129 ceph-mon[30979]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 -- 172.21.15.129:0/1175973277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 msgr2=0x7f4918110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 --2- 172.21.15.129:0/1175973277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f4918110740 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4908009900 tx=0x7f490802a410 comp rx=0 tx=0).stop 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 -- 172.21.15.129:0/1175973277 shutdown_connections 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 --2- 172.21.15.129:0/1175973277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f4918110740 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 --2- 172.21.15.129:0/1175973277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4918101150 0x7f491810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 --2- 172.21.15.129:0/1175973277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f49181007a0 0x7f4918100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 -- 172.21.15.129:0/1175973277 >> 172.21.15.129:0/1175973277 conn(0x7f49180fc610 msgr2=0x7f49180fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 -- 172.21.15.129:0/1175973277 shutdown_connections 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.272+0000 7f491cfa2640 1 -- 172.21.15.129:0/1175973277 wait complete. 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 Processor -- start 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 -- start start 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49181007a0 0x7f491819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4918101150 0x7f491819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f491819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4918113ec0 con 0x7f491810e350 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4918113d40 con 0x7f4918101150 2024-09-05T01:14:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f491cfa2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4918114040 con 0x7f49181007a0 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f491819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f491819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58384/0 (socket says 172.21.15.129:58384) 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 -- 172.21.15.129:0/2324237549 learned_addr learned my addr 172.21.15.129:0/2324237549 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f49177fe640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49181007a0 0x7f491819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4916ffd640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4918101150 0x7f491819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 -- 172.21.15.129:0/2324237549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49181007a0 msgr2=0x7f491819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49181007a0 0x7f491819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 -- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4918101150 msgr2=0x7f491819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4918101150 0x7f491819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4917fff640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491819f430 con 0x7f491810e350 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f49177fe640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f49181007a0 0x7f491819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.273+0000 7f4916ffd640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4918101150 0x7f491819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f4917fff640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f491819ed50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4908005940 tx=0x7f49080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490803f030 con 0x7f491810e350 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4908003cb0 con 0x7f491810e350 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491819f6c0 con 0x7f491810e350 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908035070 con 0x7f491810e350 2024-09-05T01:14:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.274+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49181a7000 con 0x7f491810e350 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.275+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4918062900 con 0x7f491810e350 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.276+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4908042030 con 0x7f491810e350 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4914ff9640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49040a2c90 0x7f49040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f49177fe640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49040a2c90 0x7f49040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f49080efdd0 con 0x7f491810e350 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4914ff9640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] conn(0x7f49040ad9c0 0x7f49040afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f490800b0b0 con 0x7f49040ad9c0 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f49080f0220 con 0x7f491810e350 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f49177fe640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49040a2c90 0x7f49040a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4910005c40 tx=0x7f491000a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.277+0000 7f4916ffd640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] conn(0x7f49040ad9c0 0x7f49040afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.278+0000 7f4916ffd640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] conn(0x7f49040ad9c0 0x7f49040afe00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.278+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== osd.4 v2:172.21.15.183:6808/3612040515 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f490800b0b0 con 0x7f49040ad9c0 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.292+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 --> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4918062d00 con 0x7f49040ad9c0 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.292+0000 7f4914ff9640 1 -- 172.21.15.129:0/2324237549 <== osd.4 v2:172.21.15.183:6808/3612040515 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4918062d00 con 0x7f49040ad9c0 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] conn(0x7f49040ad9c0 msgr2=0x7f49040afe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] conn(0x7f49040ad9c0 0x7f49040afe00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49040a2c90 msgr2=0x7f49040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49040a2c90 0x7f49040a5150 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4910005c40 tx=0x7f491000a430 comp rx=0 tx=0).stop 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 msgr2=0x7f491819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 --2- 172.21.15.129:0/2324237549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f491810e350 0x7f491819ed50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4908005940 tx=0x7f49080042f0 comp rx=0 tx=0).stop 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f4917fff640 1 -- 172.21.15.129:0/2324237549 reap_dead start 2024-09-05T01:14:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 shutdown_connections 2024-09-05T01:14:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 >> 172.21.15.129:0/2324237549 conn(0x7f49180fc610 msgr2=0x7f49180fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 shutdown_connections 2024-09-05T01:14:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:15.293+0000 7f491cfa2640 1 -- 172.21.15.129:0/2324237549 wait complete. 2024-09-05T01:14:15.939 INFO:teuthology.orchestra.run.smithi129.stdout:133143986239 2024-09-05T01:14:15.940 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-05T01:14:15.955 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:15.958 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:15.961 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:15.964 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:15.996 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:16.000 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:16.548 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:16.811 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:17.061 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:16 smithi129 ceph-mon[30979]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-05T01:14:17.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:16 smithi183 ceph-mon[34589]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-05T01:14:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:16 smithi194 ceph-mon[34941]: pgmap v315: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-05T01:14:17.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.581+0000 7f2f6d113640 1 -- 172.21.15.129:0/36217497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68106f10 msgr2=0x7f2f6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.581+0000 7f2f6d113640 1 --2- 172.21.15.129:0/36217497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68106f10 0x7f2f6810ea90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009900 tx=0x7f2f6002a410 comp rx=0 tx=0).stop 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.581+0000 7f2f6d113640 1 -- 172.21.15.129:0/36217497 shutdown_connections 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.581+0000 7f2f6d113640 1 --2- 172.21.15.129:0/36217497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68106f10 0x7f2f6810ea90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.581+0000 7f2f6d113640 1 --2- 172.21.15.129:0/36217497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68102790 0x7f2f681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 --2- 172.21.15.129:0/36217497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f68101de0 0x7f2f681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 -- 172.21.15.129:0/36217497 >> 172.21.15.129:0/36217497 conn(0x7f2f680fd8c0 msgr2=0x7f2f680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 -- 172.21.15.129:0/36217497 shutdown_connections 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 -- 172.21.15.129:0/36217497 wait complete. 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 Processor -- start 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.582+0000 7f2f6d113640 1 -- start start 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68101de0 0x7f2f68198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 0x7f2f681988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:17.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f68106f10 0x7f2f6819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68111e20 con 0x7f2f68102790 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68111ca0 con 0x7f2f68106f10 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6d113640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f68111fa0 con 0x7f2f68101de0 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 0x7f2f681988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 0x7f2f681988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58404/0 (socket says 172.21.15.129:58404) 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 -- 172.21.15.129:0/4200801689 learned_addr learned my addr 172.21.15.129:0/4200801689 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6759e640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f68106f10 0x7f2f6819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f66d9d640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68101de0 0x7f2f68198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 -- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68101de0 msgr2=0x7f2f68198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68101de0 0x7f2f68198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 -- 172.21.15.129:0/4200801689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f68106f10 msgr2=0x7f2f6819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f68106f10 0x7f2f6819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6819d330 con 0x7f2f68102790 2024-09-05T01:14:17.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f66d9d640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f68101de0 0x7f2f68198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.583+0000 7f2f6659c640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 0x7f2f681988c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800d770 tx=0x7f2f5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.584+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58017070 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.584+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6819d620 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.584+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f580054d0 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.584+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58004cf0 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.584+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f681a4f00 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.585+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f68062900 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.586+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2f58004e90 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.586+0000 7f2f4ffff640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f500a2c10 0x7f2f500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f66d9d640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f500a2c10 0x7f2f500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f2f580cf220 con 0x7f2f68102790 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f4ffff640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] conn(0x7f2f500ad940 0x7f2f500afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:17.592 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f58005320 con 0x7f2f500ad940 2024-09-05T01:14:17.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2f580cf670 con 0x7f2f68102790 2024-09-05T01:14:17.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f66d9d640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f500a2c10 0x7f2f500a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005c40 tx=0x7f2f5c00a430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:17.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.587+0000 7f2f6759e640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] conn(0x7f2f500ad940 0x7f2f500afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:17.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.588+0000 7f2f6759e640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] conn(0x7f2f500ad940 0x7f2f500afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:17.593 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.588+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== osd.5 v2:172.21.15.194:6808/1401637167 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f58005320 con 0x7f2f500ad940 2024-09-05T01:14:17.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.601+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 --> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f68062d80 con 0x7f2f500ad940 2024-09-05T01:14:17.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f4ffff640 1 -- 172.21.15.129:0/4200801689 <== osd.5 v2:172.21.15.194:6808/1401637167 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2f68062d80 con 0x7f2f500ad940 2024-09-05T01:14:17.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] conn(0x7f2f500ad940 msgr2=0x7f2f500afd80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] conn(0x7f2f500ad940 0x7f2f500afd80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:17.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f500a2c10 msgr2=0x7f2f500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f500a2c10 0x7f2f500a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c005c40 tx=0x7f2f5c00a430 comp rx=0 tx=0).stop 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 msgr2=0x7f2f681988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 --2- 172.21.15.129:0/4200801689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f68102790 0x7f2f681988c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800d770 tx=0x7f2f5800dc30 comp rx=0 tx=0).stop 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6759e640 1 -- 172.21.15.129:0/4200801689 reap_dead start 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 shutdown_connections 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.602+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 >> 172.21.15.129:0/4200801689 conn(0x7f2f680fd8c0 msgr2=0x7f2f680fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.603+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 shutdown_connections 2024-09-05T01:14:17.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:17.603+0000 7f2f6d113640 1 -- 172.21.15.129:0/4200801689 wait complete. 2024-09-05T01:14:17.872 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T01:14:17.873 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T01:14:17.886 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:18.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- 172.21.15.129:0/399673435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 msgr2=0x7f9c441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 --2- 172.21.15.129:0/399673435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c441029b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009900 tx=0x7f9c2c02a480 comp rx=0 tx=0).stop 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- 172.21.15.129:0/399673435 shutdown_connections 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 --2- 172.21.15.129:0/399673435 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c44102ef0 0x7f9c4410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 --2- 172.21.15.129:0/399673435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c441029b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 --2- 172.21.15.129:0/399673435 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 0x7f9c44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- 172.21.15.129:0/399673435 >> 172.21.15.129:0/399673435 conn(0x7f9c440fc7d0 msgr2=0x7f9c440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- 172.21.15.129:0/399673435 shutdown_connections 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- 172.21.15.129:0/399673435 wait complete. 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 Processor -- start 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.069+0000 7f9c4935b640 1 -- start start 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 0x7f9c44197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c44197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:18.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c44102ef0 0x7f9c4419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44110e30 con 0x7f9c44102570 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44110cb0 con 0x7f9c44100e20 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c4935b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c44110fb0 con 0x7f9c44102ef0 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c44197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c44197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58428/0 (socket says 172.21.15.129:58428) 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 -- 172.21.15.129:0/3920280307 learned_addr learned my addr 172.21.15.129:0/3920280307 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c42ffd640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 0x7f9c44197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c437fe640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c44102ef0 0x7f9c4419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 -- 172.21.15.129:0/3920280307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c44102ef0 msgr2=0x7f9c4419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c44102ef0 0x7f9c4419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 -- 172.21.15.129:0/3920280307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 msgr2=0x7f9c44197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 0x7f9c44197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4419c2c0 con 0x7f9c44102570 2024-09-05T01:14:18.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c42ffd640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c44100e20 0x7f9c44197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c427fc640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c44197850 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.070+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c046070 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.071+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4419c550 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.071+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c2c003aa0 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.071+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c035050 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.071+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c44069f70 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.071+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c4406a520 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.073+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9c2c049070 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.073+0000 7f9c3bfff640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c300a2c10 0x7f9c300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c42ffd640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c300a2c10 0x7f9c300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9092+0+0 (secure 0 0 0) 0x7f9c2c0f1750 con 0x7f9c44102570 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c3bfff640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] conn(0x7f9c300adde0 0x7f9c300b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:18.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c2c001d30 con 0x7f9c300adde0 2024-09-05T01:14:18.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9c2c0f1ba0 con 0x7f9c44102570 2024-09-05T01:14:18.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c437fe640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] conn(0x7f9c300adde0 0x7f9c300b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:18.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c42ffd640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c300a2c10 0x7f9c300a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005fd0 tx=0x7f9c34009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:18.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.074+0000 7f9c437fe640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] conn(0x7f9c300adde0 0x7f9c300b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:18.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.075+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== osd.6 v2:172.21.15.129:6818/2103992741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c2c001d30 con 0x7f9c300adde0 2024-09-05T01:14:18.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.088+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 --> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c4406a920 con 0x7f9c300adde0 2024-09-05T01:14:18.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.088+0000 7f9c3bfff640 1 -- 172.21.15.129:0/3920280307 <== osd.6 v2:172.21.15.129:6818/2103992741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9c4406a920 con 0x7f9c300adde0 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] conn(0x7f9c300adde0 msgr2=0x7f9c300b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] conn(0x7f9c300adde0 0x7f9c300b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c300a2c10 msgr2=0x7f9c300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c300a2c10 0x7f9c300a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c34005fd0 tx=0x7f9c34009600 comp rx=0 tx=0).stop 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 msgr2=0x7f9c44197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 --2- 172.21.15.129:0/3920280307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c44102570 0x7f9c44197850 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006010 tx=0x7f9c2c004270 comp rx=0 tx=0).stop 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c437fe640 1 -- 172.21.15.129:0/3920280307 reap_dead start 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 shutdown_connections 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 >> 172.21.15.129:0/3920280307 conn(0x7f9c440fc7d0 msgr2=0x7f9c440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 shutdown_connections 2024-09-05T01:14:18.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:18.089+0000 7f9c4935b640 1 -- 172.21.15.129:0/3920280307 wait complete. 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T01:14:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T01:14:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-05T01:14:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-09-05T01:14:18.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-05T01:14:18.905 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T01:14:18.906 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:18 smithi129 ceph-mon[30979]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T01:14:18.936 INFO:teuthology.orchestra.run.smithi129.stdout:154618822713 2024-09-05T01:14:18.937 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-05T01:14:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-05T01:14:19.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-05T01:14:19.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T01:14:19.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T01:14:19.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:18 smithi183 ceph-mon[34589]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T01:14:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: pgmap v316: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-05T01:14:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 7]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-05T01:14:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:18 smithi194 ceph-mon[34941]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T01:14:19.457 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 -- 172.21.15.129:0/4104864059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010102d40 msgr2=0x7f6010103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 --2- 172.21.15.129:0/4104864059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010102d40 0x7f6010103120 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002a480 comp rx=0 tx=0).stop 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 -- 172.21.15.129:0/4104864059 shutdown_connections 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 --2- 172.21.15.129:0/4104864059 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f601010c170 0x7f601010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 --2- 172.21.15.129:0/4104864059 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 0x7f601010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 --2- 172.21.15.129:0/4104864059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010102d40 0x7f6010103120 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 -- 172.21.15.129:0/4104864059 >> 172.21.15.129:0/4104864059 conn(0x7f60100fc7a0 msgr2=0x7f60100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 -- 172.21.15.129:0/4104864059 shutdown_connections 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.433+0000 7f6014ebf640 1 -- 172.21.15.129:0/4104864059 wait complete. 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 Processor -- start 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 -- start start 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6010102d40 0x7f6010198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:19.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 0x7f6010198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 0x7f601019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111e20 con 0x7f601010c170 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111ca0 con 0x7f6010102d40 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f6014ebf640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111fa0 con 0x7f6010103660 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f600ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 0x7f601019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f600ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 0x7f601019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58468/0 (socket says 172.21.15.129:58468) 2024-09-05T01:14:19.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f600ed9d640 1 -- 172.21.15.129:0/1289973934 learned_addr learned my addr 172.21.15.129:0/1289973934 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f600e59c640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6010102d40 0x7f6010198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.434+0000 7f600dd9b640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 0x7f6010198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 -- 172.21.15.129:0/1289973934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 msgr2=0x7f6010198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 0x7f6010198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 -- 172.21.15.129:0/1289973934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6010102d40 msgr2=0x7f6010198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6010102d40 0x7f6010198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601019d300 con 0x7f601010c170 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600dd9b640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6010103660 0x7f6010198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f600ed9d640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 0x7f601019cc20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f600800eaa0 tx=0x7f600800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600800cdb0 con 0x7f601010c170 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601019d5f0 con 0x7f601010c170 2024-09-05T01:14:19.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a4ed0 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6008005020 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.435+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60080116d0 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.436+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ff0001030 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.437+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6008004a40 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f5fff7fe640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff80a2c90 0x7f5ff80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f600e59c640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff80a2c90 0x7f5ff80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f60080cfa40 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f5fff7fe640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] conn(0x7f5ff80ade60 0x7f5ff80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6008014ec0 con 0x7f5ff80ade60 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f60080cfe90 con 0x7f601010c170 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f600dd9b640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] conn(0x7f5ff80ade60 0x7f5ff80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.438+0000 7f600e59c640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff80a2c90 0x7f5ff80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6000031440 tx=0x7f6000005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.439+0000 7f600dd9b640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] conn(0x7f5ff80ade60 0x7f5ff80b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:19.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.439+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== osd.9 v2:172.21.15.129:6826/1276233090 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6008014ec0 con 0x7f5ff80ade60 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 --> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ff0002dd0 con 0x7f5ff80ade60 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f5fff7fe640 1 -- 172.21.15.129:0/1289973934 <== osd.9 v2:172.21.15.129:6826/1276233090 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ff0002dd0 con 0x7f5ff80ade60 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] conn(0x7f5ff80ade60 msgr2=0x7f5ff80b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] conn(0x7f5ff80ade60 0x7f5ff80b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff80a2c90 msgr2=0x7f5ff80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff80a2c90 0x7f5ff80a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6000031440 tx=0x7f6000005b50 comp rx=0 tx=0).stop 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 msgr2=0x7f601019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 --2- 172.21.15.129:0/1289973934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f601010c170 0x7f601019cc20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f600800eaa0 tx=0x7f600800ef60 comp rx=0 tx=0).stop 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f600ed9d640 1 -- 172.21.15.129:0/1289973934 reap_dead start 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 shutdown_connections 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 >> 172.21.15.129:0/1289973934 conn(0x7f60100fc7a0 msgr2=0x7f60100fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 shutdown_connections 2024-09-05T01:14:19.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.453+0000 7f6014ebf640 1 -- 172.21.15.129:0/1289973934 wait complete. 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 -- 172.21.15.129:0/2016191185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8100a80 msgr2=0x7efdf8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 --2- 172.21.15.129:0/2016191185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8100a80 0x7efdf8100ee0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efdec009930 tx=0x7efdec02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 -- 172.21.15.129:0/2016191185 shutdown_connections 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 --2- 172.21.15.129:0/2016191185 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 0x7efdf810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 --2- 172.21.15.129:0/2016191185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8100a80 0x7efdf8100ee0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 --2- 172.21.15.129:0/2016191185 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdf8106a80 0x7efdf8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 -- 172.21.15.129:0/2016191185 >> 172.21.15.129:0/2016191185 conn(0x7efdf80fc7a0 msgr2=0x7efdf80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 -- 172.21.15.129:0/2016191185 shutdown_connections 2024-09-05T01:14:20.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 -- 172.21.15.129:0/2016191185 wait complete. 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.070+0000 7efdfe065640 1 Processor -- start 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 -- start start 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdf8100a80 0x7efdf8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 0x7efdf8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 0x7efdf819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110dd0 con 0x7efdf8106a80 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110c50 con 0x7efdf8101420 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdfe065640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efdf8110f50 con 0x7efdf8100a80 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 0x7efdf819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 0x7efdf819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58500/0 (socket says 172.21.15.129:58500) 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 -- 172.21.15.129:0/3286969337 learned_addr learned my addr 172.21.15.129:0/3286969337 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:20.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf6ffd640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 0x7efdf8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf77fe640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdf8100a80 0x7efdf8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 -- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdf8100a80 msgr2=0x7efdf8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efdf8100a80 0x7efdf8198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 -- 172.21.15.129:0/3286969337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 msgr2=0x7efdf8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 0x7efdf8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf819d4b0 con 0x7efdf8106a80 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf6ffd640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdf8101420 0x7efdf8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.071+0000 7efdf7fff640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 0x7efdf819cdd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b970 tx=0x7efdf000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0019070 con 0x7efdf8106a80 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf0014440 con 0x7efdf8106a80 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf819d7a0 con 0x7efdf8106a80 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00075d0 con 0x7efdf8106a80 2024-09-05T01:14:20.083 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81a5080 con 0x7efdf8106a80 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.072+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdf8062980 con 0x7efdf8106a80 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.074+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efdf001c070 con 0x7efdf8106a80 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.074+0000 7efdf4ff9640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efde00a2c10 0x7efde00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf77fe640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efde00a2c10 0x7efde00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7efdf00cf250 con 0x7efdf8106a80 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf4ff9640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] conn(0x7efde00adde0 0x7efde00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdf00047d0 con 0x7efde00adde0 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7efdf00cf6a0 con 0x7efdf8106a80 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf77fe640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efde00a2c10 0x7efde00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efde800a810 tx=0x7efde8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.075+0000 7efdf6ffd640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] conn(0x7efde00adde0 0x7efde00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.076+0000 7efdf6ffd640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] conn(0x7efde00adde0 0x7efde00b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.076+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== osd.2 v2:172.21.15.194:6800/968743504 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdf00047d0 con 0x7efde00adde0 2024-09-05T01:14:20.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.090+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 --> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdf8062e00 con 0x7efde00adde0 2024-09-05T01:14:20.092 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:19 smithi129 ceph-mon[30979]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T01:14:20.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdf4ff9640 1 -- 172.21.15.129:0/3286969337 <== osd.2 v2:172.21.15.194:6800/968743504 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efdf8062e00 con 0x7efde00adde0 2024-09-05T01:14:20.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] conn(0x7efde00adde0 msgr2=0x7efde00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] conn(0x7efde00adde0 0x7efde00b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efde00a2c10 msgr2=0x7efde00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efde00a2c10 0x7efde00a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efde800a810 tx=0x7efde8008040 comp rx=0 tx=0).stop 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 msgr2=0x7efdf819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 --2- 172.21.15.129:0/3286969337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdf8106a80 0x7efdf819cdd0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b970 tx=0x7efdf000be30 comp rx=0 tx=0).stop 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdf7fff640 1 -- 172.21.15.129:0/3286969337 reap_dead start 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 shutdown_connections 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 >> 172.21.15.129:0/3286969337 conn(0x7efdf80fc7a0 msgr2=0x7efdf806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 shutdown_connections 2024-09-05T01:14:20.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:20.091+0000 7efdfe065640 1 -- 172.21.15.129:0/3286969337 wait complete. 2024-09-05T01:14:20.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:19 smithi183 ceph-mon[34589]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T01:14:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:19 smithi194 ceph-mon[34941]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T01:14:20.572 INFO:teuthology.orchestra.run.smithi129.stdout:184683593781 2024-09-05T01:14:20.572 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-05T01:14:20.878 INFO:teuthology.orchestra.run.smithi129.stdout:266287972390 2024-09-05T01:14:20.878 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-05T01:14:20.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 -- 172.21.15.129:0/3140321713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 msgr2=0x7f94c0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/3140321713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0100ee0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f94a8009930 tx=0x7f94a802a420 comp rx=0 tx=0).stop 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 -- 172.21.15.129:0/3140321713 shutdown_connections 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/3140321713 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c0101420 0x7f94c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/3140321713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0100ee0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/3140321713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 0x7f94c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 -- 172.21.15.129:0/3140321713 >> 172.21.15.129:0/3140321713 conn(0x7f94c00fc7a0 msgr2=0x7f94c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 -- 172.21.15.129:0/3140321713 shutdown_connections 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.330+0000 7f94c4b4d640 1 -- 172.21.15.129:0/3140321713 wait complete. 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 Processor -- start 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 -- start start 2024-09-05T01:14:20.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c0101420 0x7f94c01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 0x7f94c019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0110cf0 con 0x7f94c0100a80 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0110b70 con 0x7f94c0106a80 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94c4b4d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94c0110e70 con 0x7f94c0101420 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58448/0 (socket says 172.21.15.129:58448) 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 -- 172.21.15.129:0/410964905 learned_addr learned my addr 172.21.15.129:0/410964905 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94b7fff640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c0101420 0x7f94c01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94bed9d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 0x7f94c019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 -- 172.21.15.129:0/410964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c0101420 msgr2=0x7f94c01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c0101420 0x7f94c01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 -- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 msgr2=0x7f94c019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 0x7f94c019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94be59c640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c019d320 con 0x7f94c0100a80 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.331+0000 7f94bed9d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c0106a80 0x7f94c019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94be59c640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0198370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c920 tx=0x7f94b000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0007c70 con 0x7f94c0100a80 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c019d610 con 0x7f94c0100a80 2024-09-05T01:14:20.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b0007e10 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0018820 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c01a4ef0 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.332+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94c00fd190 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.334+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f94b00189c0 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.334+0000 7f94b77fe640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94ac0a2c10 0x7f94ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b7fff640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94ac0a2c10 0x7f94ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f94b00cfab0 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b77fe640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] conn(0x7f94ac0addd0 0x7f94ac0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94b0015e70 con 0x7f94ac0addd0 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f94b00cff00 con 0x7f94c0100a80 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94b7fff640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94ac0a2c10 0x7f94ac0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8032c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94bed9d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] conn(0x7f94ac0addd0 0x7f94ac0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:20.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.335+0000 7f94bed9d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] conn(0x7f94ac0addd0 0x7f94ac0b0270 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.336+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== osd.7 v2:172.21.15.183:6816/1669309278 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94b0015e70 con 0x7f94ac0addd0 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.350+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 --> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94c01021c0 con 0x7f94ac0addd0 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.350+0000 7f94b77fe640 1 -- 172.21.15.129:0/410964905 <== osd.7 v2:172.21.15.183:6816/1669309278 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94c01021c0 con 0x7f94ac0addd0 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] conn(0x7f94ac0addd0 msgr2=0x7f94ac0b0270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] conn(0x7f94ac0addd0 0x7f94ac0b0270 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94ac0a2c10 msgr2=0x7f94ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94ac0a2c10 0x7f94ac0a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f94a8000c00 tx=0x7f94a8032c80 comp rx=0 tx=0).stop 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 msgr2=0x7f94c0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 --2- 172.21.15.129:0/410964905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c0100a80 0x7f94c0198370 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c920 tx=0x7f94b000cde0 comp rx=0 tx=0).stop 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94bed9d640 1 -- 172.21.15.129:0/410964905 reap_dead start 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 shutdown_connections 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 >> 172.21.15.129:0/410964905 conn(0x7f94c00fc7a0 msgr2=0x7f94c01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 shutdown_connections 2024-09-05T01:14:20.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:19.351+0000 7f94c4b4d640 1 -- 172.21.15.129:0/410964905 wait complete. 2024-09-05T01:14:21.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:20 smithi183 ceph-mon[34589]: pgmap v319: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-09-05T01:14:21.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:20 smithi183 ceph-mon[34589]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T01:14:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:20 smithi129 ceph-mon[30979]: pgmap v319: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-09-05T01:14:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:20 smithi129 ceph-mon[30979]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T01:14:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:20 smithi194 ceph-mon[34941]: pgmap v319: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 383 B/s wr, 7 op/s 2024-09-05T01:14:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:20 smithi194 ceph-mon[34941]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-05T01:14:22.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 -- 172.21.15.129:0/1647564913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 msgr2=0x7f74781067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 --2- 172.21.15.129:0/1647564913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f74781067f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7460009900 tx=0x7f746002a410 comp rx=0 tx=0).stop 2024-09-05T01:14:22.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 -- 172.21.15.129:0/1647564913 shutdown_connections 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 --2- 172.21.15.129:0/1647564913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7478106f20 0x7f747810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 --2- 172.21.15.129:0/1647564913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f74781067f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 --2- 172.21.15.129:0/1647564913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 0x7f74781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 -- 172.21.15.129:0/1647564913 >> 172.21.15.129:0/1647564913 conn(0x7f74780fd8f0 msgr2=0x7f74780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 -- 172.21.15.129:0/1647564913 shutdown_connections 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.033+0000 7f747e1cb640 1 -- 172.21.15.129:0/1647564913 wait complete. 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 Processor -- start 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 -- start start 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 0x7f74780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f7478074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:22.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7478106f20 0x7f7478074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7478111ed0 con 0x7f74781027a0 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7478111d50 con 0x7f7478106f20 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f747e1cb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7478112050 con 0x7f7478101df0 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f746effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f7478074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f746effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f7478074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58520/0 (socket says 172.21.15.129:58520) 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f746effd640 1 -- 172.21.15.129:0/791441498 learned_addr learned my addr 172.21.15.129:0/791441498 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.034+0000 7f74777fe640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 0x7f74780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 -- 172.21.15.129:0/791441498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 msgr2=0x7f74780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 0x7f74780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 -- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7478106f20 msgr2=0x7f7478074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f7477fff640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7478106f20 0x7f7478074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7478106f20 0x7f7478074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74781a69d0 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f74777fe640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7478101df0 0x7f74780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f746effd640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f7478074a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7460006010 tx=0x7f7460004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746003f030 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74781a6c60 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74600035f0 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460035070 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.035+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74781a7140 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.036+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7478062880 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.037+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7460003b10 con 0x7f74781027a0 2024-09-05T01:14:22.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74757fa640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74640a2c10 0x7f74640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74777fe640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74640a2c10 0x7f74640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f74600f0810 con 0x7f74781027a0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74757fa640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] conn(0x7f74640adde0 0x7f74640b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74640b0950 con 0x7f74640adde0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.038+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f74600f0c60 con 0x7f74781027a0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.039+0000 7f74777fe640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74640a2c10 0x7f74640a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7468009930 tx=0x7f7468008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.039+0000 7f7477fff640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] conn(0x7f74640adde0 0x7f74640b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.039+0000 7f7477fff640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] conn(0x7f74640adde0 0x7f74640b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:22.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.040+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== osd.1 v2:172.21.15.183:6800/63466833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74640b0950 con 0x7f74640adde0 2024-09-05T01:14:22.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.053+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 --> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7478062d00 con 0x7f74640adde0 2024-09-05T01:14:22.055 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f74757fa640 1 -- 172.21.15.129:0/791441498 <== osd.1 v2:172.21.15.183:6800/63466833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7478062d00 con 0x7f74640adde0 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] conn(0x7f74640adde0 msgr2=0x7f74640b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] conn(0x7f74640adde0 0x7f74640b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74640a2c10 msgr2=0x7f74640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74640a2c10 0x7f74640a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7468009930 tx=0x7f7468008040 comp rx=0 tx=0).stop 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 msgr2=0x7f7478074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:22.056 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 --2- 172.21.15.129:0/791441498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f74781027a0 0x7f7478074a30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7460006010 tx=0x7f7460004270 comp rx=0 tx=0).stop 2024-09-05T01:14:22.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f7477fff640 1 -- 172.21.15.129:0/791441498 reap_dead start 2024-09-05T01:14:22.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 shutdown_connections 2024-09-05T01:14:22.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 >> 172.21.15.129:0/791441498 conn(0x7f74780fd8f0 msgr2=0x7f74780feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:22.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.054+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 shutdown_connections 2024-09-05T01:14:22.057 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:22.055+0000 7f747e1cb640 1 -- 172.21.15.129:0/791441498 wait complete. 2024-09-05T01:14:23.108 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:22 smithi129 ceph-mon[30979]: pgmap v320: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-05T01:14:23.150 INFO:teuthology.orchestra.run.smithi129.stdout:77309411402 2024-09-05T01:14:23.150 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-05T01:14:23.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:22 smithi183 ceph-mon[34589]: pgmap v320: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-05T01:14:23.169 INFO:teuthology.orchestra.run.smithi129.stdout:206158430256 2024-09-05T01:14:23.170 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-05T01:14:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:22 smithi194 ceph-mon[34941]: pgmap v320: 129 pgs: 1 activating, 2 peering, 126 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-05T01:14:23.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 -- 172.21.15.129:0/948662030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1410c170 msgr2=0x7f1a1410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/948662030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1410c170 0x7f1a1410e610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009ca0 tx=0x7f1a0c02a950 comp rx=0 tx=0).stop 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 -- 172.21.15.129:0/948662030 shutdown_connections 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/948662030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a1410c170 0x7f1a1410e610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/948662030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a14103660 0x7f1a1410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/948662030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14102d40 0x7f1a14103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 -- 172.21.15.129:0/948662030 >> 172.21.15.129:0/948662030 conn(0x7f1a140fc7a0 msgr2=0x7f1a140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 -- 172.21.15.129:0/948662030 shutdown_connections 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.334+0000 7f1a1bf72640 1 -- 172.21.15.129:0/948662030 wait complete. 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 Processor -- start 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 -- start start 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 0x7f1a14198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14103660 0x7f1a14198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:23.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1410c170 0x7f1a1419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111e20 con 0x7f1a14102d40 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111ca0 con 0x7f1a14103660 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a1bf72640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a14111fa0 con 0x7f1a1410c170 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a19d0e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 0x7f1a14198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a19d0e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 0x7f1a14198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48586/0 (socket says 172.21.15.129:48586) 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.335+0000 7f1a19d0e640 1 -- 172.21.15.129:0/3313797102 learned_addr learned my addr 172.21.15.129:0/3313797102 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a1a50f640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1410c170 0x7f1a1419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a1950d640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14103660 0x7f1a14198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 -- 172.21.15.129:0/3313797102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1410c170 msgr2=0x7f1a1419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a1410c170 0x7f1a1419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 -- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14103660 msgr2=0x7f1a14198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14103660 0x7f1a14198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1419d2d0 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a1950d640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a14103660 0x7f1a14198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a19d0e640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 0x7f1a14198320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e9f0 tx=0x7f1a1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000cd80 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a100108e0 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1419d5c0 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10018920 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.336+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a141008d0 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.337+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a14062980 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.340+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1a10010a90 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.341+0000 7f1a0affd640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a000a2c10 0x7f1a000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.341+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1a100d0040 con 0x7f1a14102d40 2024-09-05T01:14:23.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.341+0000 7f1a1950d640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a000a2c10 0x7f1a000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.341+0000 7f1a0affd640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] conn(0x7f1a000adde0 0x7f1a000b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.341+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a000b0950 con 0x7f1a000adde0 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.342+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1a100d0490 con 0x7f1a14102d40 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.342+0000 7f1a1a50f640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] conn(0x7f1a000adde0 0x7f1a000b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.342+0000 7f1a1950d640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a000a2c10 0x7f1a000a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005d70 tx=0x7f1a04005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.342+0000 7f1a1a50f640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] conn(0x7f1a000adde0 0x7f1a000b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:23.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.343+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== osd.10 v2:172.21.15.183:6824/1458641027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a000b0950 con 0x7f1a000adde0 2024-09-05T01:14:23.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.359+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 --> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a14062e00 con 0x7f1a000adde0 2024-09-05T01:14:23.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a0affd640 1 -- 172.21.15.129:0/3313797102 <== osd.10 v2:172.21.15.183:6824/1458641027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1a14062e00 con 0x7f1a000adde0 2024-09-05T01:14:23.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] conn(0x7f1a000adde0 msgr2=0x7f1a000b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] conn(0x7f1a000adde0 0x7f1a000b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a000a2c10 msgr2=0x7f1a000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a000a2c10 0x7f1a000a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005d70 tx=0x7f1a04005d00 comp rx=0 tx=0).stop 2024-09-05T01:14:23.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 msgr2=0x7f1a14198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1bf72640 1 --2- 172.21.15.129:0/3313797102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a14102d40 0x7f1a14198320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a1000e9f0 tx=0x7f1a1000eeb0 comp rx=0 tx=0).stop 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.360+0000 7f1a1a50f640 1 -- 172.21.15.129:0/3313797102 reap_dead start 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.361+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 shutdown_connections 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.361+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 >> 172.21.15.129:0/3313797102 conn(0x7f1a140fc7a0 msgr2=0x7f1a140fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.361+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 shutdown_connections 2024-09-05T01:14:23.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:23.361+0000 7f1a1bf72640 1 -- 172.21.15.129:0/3313797102 wait complete. 2024-09-05T01:14:23.853 INFO:teuthology.orchestra.run.smithi129.stdout:55834574927 2024-09-05T01:14:23.854 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-05T01:14:24.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 -- 172.21.15.129:0/4224301691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 msgr2=0x7f69641067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 --2- 172.21.15.129:0/4224301691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f69641067f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f694c009900 tx=0x7f694c02a410 comp rx=0 tx=0).stop 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 -- 172.21.15.129:0/4224301691 shutdown_connections 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 --2- 172.21.15.129:0/4224301691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6964106f20 0x7f696410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 --2- 172.21.15.129:0/4224301691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f69641067f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 --2- 172.21.15.129:0/4224301691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6964101df0 0x7f69641021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 -- 172.21.15.129:0/4224301691 >> 172.21.15.129:0/4224301691 conn(0x7f69640fd8b0 msgr2=0x7f69640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 -- 172.21.15.129:0/4224301691 shutdown_connections 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.005+0000 7f696952b640 1 -- 172.21.15.129:0/4224301691 wait complete. 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 Processor -- start 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 -- start start 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6964101df0 0x7f69641983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.032 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f6964198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6964106f20 0x7f696419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111e30 con 0x7f69641027a0 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111cb0 con 0x7f6964106f20 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f696952b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6964111fb0 con 0x7f6964101df0 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f6964198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f6964198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48608/0 (socket says 172.21.15.129:48608) 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 -- 172.21.15.129:0/3819419839 learned_addr learned my addr 172.21.15.129:0/3819419839 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f6962ffd640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6964101df0 0x7f69641983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f69637fe640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6964106f20 0x7f696419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 -- 172.21.15.129:0/3819419839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6964101df0 msgr2=0x7f69641983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6964101df0 0x7f69641983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.033 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 -- 172.21.15.129:0/3819419839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6964106f20 msgr2=0x7f696419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6964106f20 0x7f696419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.006+0000 7f695a7fc640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696419d3a0 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f695a7fc640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f6964198930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f694c005940 tx=0x7f694c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c03f030 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696419d630 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f694c003aa0 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c035070 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.007+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69641a4f70 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.008+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6964062900 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.009+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f694c042030 con 0x7f69641027a0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6960ff9640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f69500a2c10 0x7f69500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6962ffd640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f69500a2c10 0x7f69500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f694c0f0020 con 0x7f69641027a0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6960ff9640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] conn(0x7f69500adde0 0x7f69500b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69500b0950 con 0x7f69500adde0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f694c0f0470 con 0x7f69641027a0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f69637fe640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] conn(0x7f69500adde0 0x7f69500b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.010+0000 7f6962ffd640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f69500a2c10 0x7f69500a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f695400bbe0 tx=0x7f695400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.011+0000 7f69637fe640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] conn(0x7f69500adde0 0x7f69500b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.012+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== osd.0 v2:172.21.15.129:6802/2948481734 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69500b0950 con 0x7f69500adde0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.025+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 --> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6964062d00 con 0x7f69500adde0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.025+0000 7f6960ff9640 1 -- 172.21.15.129:0/3819419839 <== osd.0 v2:172.21.15.129:6802/2948481734 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6964062d00 con 0x7f69500adde0 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] conn(0x7f69500adde0 msgr2=0x7f69500b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.035 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] conn(0x7f69500adde0 0x7f69500b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f69500a2c10 msgr2=0x7f69500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f69500a2c10 0x7f69500a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f695400bbe0 tx=0x7f695400b390 comp rx=0 tx=0).stop 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 msgr2=0x7f6964198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 --2- 172.21.15.129:0/3819419839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f69641027a0 0x7f6964198930 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f694c005940 tx=0x7f694c004270 comp rx=0 tx=0).stop 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f69637fe640 1 -- 172.21.15.129:0/3819419839 reap_dead start 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 shutdown_connections 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 >> 172.21.15.129:0/3819419839 conn(0x7f69640fd8b0 msgr2=0x7f69640fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:24.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 shutdown_connections 2024-09-05T01:14:24.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.026+0000 7f696952b640 1 -- 172.21.15.129:0/3819419839 wait complete. 2024-09-05T01:14:24.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:23 smithi183 ceph-mon[34589]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T01:14:24.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:23 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:14:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:23 smithi129 ceph-mon[30979]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T01:14:24.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:23 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:14:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:23 smithi194 ceph-mon[34941]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-05T01:14:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:23 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 -- 172.21.15.129:0/3484535029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34101420 msgr2=0x7f4f3410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 --2- 172.21.15.129:0/3484535029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34101420 0x7f4f3410d950 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c009cd0 tx=0x7f4f2c02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 -- 172.21.15.129:0/3484535029 shutdown_connections 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 --2- 172.21.15.129:0/3484535029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34101420 0x7f4f3410d950 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 --2- 172.21.15.129:0/3484535029 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f34100a80 0x7f4f34100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 --2- 172.21.15.129:0/3484535029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 0x7f4f34106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 -- 172.21.15.129:0/3484535029 >> 172.21.15.129:0/3484535029 conn(0x7f4f340fc7a0 msgr2=0x7f4f340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 -- 172.21.15.129:0/3484535029 shutdown_connections 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.538+0000 7f4f39d84640 1 -- 172.21.15.129:0/3484535029 wait complete. 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 Processor -- start 2024-09-05T01:14:24.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 -- start start 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 0x7f4f340fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f34101420 0x7f4f340fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 0x7f4f340fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34110d30 con 0x7f4f34100a80 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34110bb0 con 0x7f4f34106a80 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f39d84640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f34110eb0 con 0x7f4f34101420 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 0x7f4f340fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 0x7f4f340fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48632/0 (socket says 172.21.15.129:48632) 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.539+0000 7f4f337fe640 1 -- 172.21.15.129:0/4194775629 learned_addr learned my addr 172.21.15.129:0/4194775629 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f33fff640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 0x7f4f340fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 -- 172.21.15.129:0/4194775629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f34101420 msgr2=0x7f4f340fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f32ffd640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f34101420 0x7f4f340fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f34101420 0x7f4f340fd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 -- 172.21.15.129:0/4194775629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 msgr2=0x7f4f340fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 0x7f4f340fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f341a6840 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f33fff640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f34106a80 0x7f4f340fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f337fe640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 0x7f4f340fce50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400e9f0 tx=0x7f4f2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2400cd80 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f240108e0 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f341a6b30 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24018920 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.540+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f341a70c0 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.541+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f340628c0 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.542+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4f24010a90 con 0x7f4f34100a80 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f30ff9640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f1c0a2c90 0x7f4f1c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f32ffd640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f1c0a2c90 0x7f4f1c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f4f240d0070 con 0x7f4f34100a80 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f30ff9640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] conn(0x7f4f1c0ade60 0x7f4f1c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f1c0b09d0 con 0x7f4f1c0ade60 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f4f240d04c0 con 0x7f4f34100a80 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f33fff640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] conn(0x7f4f1c0ade60 0x7f4f1c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.543+0000 7f4f32ffd640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f1c0a2c90 0x7f4f1c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f340fe8e0 tx=0x7f4f28005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.544+0000 7f4f33fff640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] conn(0x7f4f1c0ade60 0x7f4f1c0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.544+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== osd.3 v2:172.21.15.129:6810/171450732 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f1c0b09d0 con 0x7f4f1c0ade60 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 --> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f34062cf0 con 0x7f4f1c0ade60 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f30ff9640 1 -- 172.21.15.129:0/4194775629 <== osd.3 v2:172.21.15.129:6810/171450732 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f34062cf0 con 0x7f4f1c0ade60 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] conn(0x7f4f1c0ade60 msgr2=0x7f4f1c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] conn(0x7f4f1c0ade60 0x7f4f1c0b0300 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f1c0a2c90 msgr2=0x7f4f1c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f1c0a2c90 0x7f4f1c0a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4f340fe8e0 tx=0x7f4f28005c60 comp rx=0 tx=0).stop 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 msgr2=0x7f4f340fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 --2- 172.21.15.129:0/4194775629 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f34100a80 0x7f4f340fce50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400e9f0 tx=0x7f4f2400eeb0 comp rx=0 tx=0).stop 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f33fff640 1 -- 172.21.15.129:0/4194775629 reap_dead start 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 shutdown_connections 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 >> 172.21.15.129:0/4194775629 conn(0x7f4f340fc7a0 msgr2=0x7f4f3406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.558+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 shutdown_connections 2024-09-05T01:14:24.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:24.559+0000 7f4f39d84640 1 -- 172.21.15.129:0/4194775629 wait complete. 2024-09-05T01:14:25.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:24 smithi183 ceph-mon[34589]: pgmap v321: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-05T01:14:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:24 smithi129 ceph-mon[30979]: pgmap v321: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-05T01:14:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:24 smithi194 ceph-mon[34941]: pgmap v321: 129 pgs: 1 activating, 128 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 7 op/s 2024-09-05T01:14:25.311 INFO:teuthology.orchestra.run.smithi129.stdout:34359738452 2024-09-05T01:14:25.311 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-05T01:14:25.495 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:25.496 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:26.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.998+0000 7fc46848c640 1 -- 172.21.15.129:0/3192581195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 msgr2=0x7fc4601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.998+0000 7fc46848c640 1 --2- 172.21.15.129:0/3192581195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601029b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc454009900 tx=0x7fc45402a480 comp rx=0 tx=0).stop 2024-09-05T01:14:26.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- 172.21.15.129:0/3192581195 shutdown_connections 2024-09-05T01:14:26.045 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- 172.21.15.129:0/3192581195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc460102ef0 0x7fc46010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- 172.21.15.129:0/3192581195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601029b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- 172.21.15.129:0/3192581195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460100e20 0x7fc460101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- 172.21.15.129:0/3192581195 >> 172.21.15.129:0/3192581195 conn(0x7fc4600fc7d0 msgr2=0x7fc4600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- 172.21.15.129:0/3192581195 shutdown_connections 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- 172.21.15.129:0/3192581195 wait complete. 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 Processor -- start 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- start start 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc460100e20 0x7fc4601983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460102ef0 0x7fc46019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc460110e30 con 0x7fc460102570 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc460110cb0 con 0x7fc460100e20 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:25.999+0000 7fc46848c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc460110fb0 con 0x7fc460102ef0 2024-09-05T01:14:26.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48654/0 (socket says 172.21.15.129:48654) 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 -- 172.21.15.129:0/2512814987 learned_addr learned my addr 172.21.15.129:0/2512814987 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc466228640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc460100e20 0x7fc4601983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc466a29640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460102ef0 0x7fc46019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 -- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460102ef0 msgr2=0x7fc46019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460102ef0 0x7fc46019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 -- 172.21.15.129:0/2512814987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc460100e20 msgr2=0x7fc4601983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc460100e20 0x7fc4601983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46019d360 con 0x7fc460102570 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc466a29640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc460102ef0 0x7fc46019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:26.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc465a27640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601988f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc454006010 tx=0x7fc4540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45403f070 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46019d5f0 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc454003aa0 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454035030 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.000+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a4f30 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.001+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc460107260 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.002+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc454042070 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc4537fe640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc44c0a2c10 0x7fc44c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc466228640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc44c0a2c10 0x7fc44c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc4540f04c0 con 0x7fc460102570 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc4537fe640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] conn(0x7fc44c0adde0 0x7fc44c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc454038e90 con 0x7fc44c0adde0 2024-09-05T01:14:26.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.003+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc4540f0910 con 0x7fc460102570 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.004+0000 7fc466228640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc44c0a2c10 0x7fc44c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0058c0 tx=0x7fc45c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.004+0000 7fc466a29640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] conn(0x7fc44c0adde0 0x7fc44c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.004+0000 7fc466a29640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] conn(0x7fc44c0adde0 0x7fc44c0b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.005+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== osd.11 v2:172.21.15.194:6824/3769265743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc454038e90 con 0x7fc44c0adde0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.018+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 --> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc4601021b0 con 0x7fc44c0adde0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.019+0000 7fc4537fe640 1 -- 172.21.15.129:0/2512814987 <== osd.11 v2:172.21.15.194:6824/3769265743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc4601021b0 con 0x7fc44c0adde0 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.019+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] conn(0x7fc44c0adde0 msgr2=0x7fc44c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.019+0000 7fc46848c640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] conn(0x7fc44c0adde0 0x7fc44c0b0280 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.019+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc44c0a2c10 msgr2=0x7fc44c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc44c0a2c10 0x7fc44c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0058c0 tx=0x7fc45c00a2d0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 msgr2=0x7fc4601988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 --2- 172.21.15.129:0/2512814987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc460102570 0x7fc4601988f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc454006010 tx=0x7fc4540042f0 comp rx=0 tx=0).stop 2024-09-05T01:14:26.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc466a29640 1 -- 172.21.15.129:0/2512814987 reap_dead start 2024-09-05T01:14:26.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 shutdown_connections 2024-09-05T01:14:26.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 >> 172.21.15.129:0/2512814987 conn(0x7fc4600fc7d0 msgr2=0x7fc460069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:26.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 shutdown_connections 2024-09-05T01:14:26.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:26.020+0000 7fc46848c640 1 -- 172.21.15.129:0/2512814987 wait complete. 2024-09-05T01:14:26.670 INFO:teuthology.orchestra.run.smithi129.stdout:287762808866 2024-09-05T01:14:26.670 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-05T01:14:27.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:26 smithi183 ceph-mon[34589]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-09-05T01:14:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:26 smithi129 ceph-mon[30979]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-09-05T01:14:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:26 smithi194 ceph-mon[34941]: pgmap v322: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6 op/s 2024-09-05T01:14:27.358 INFO:teuthology.orchestra.run.smithi129.stdout:317827579934 2024-09-05T01:14:27.359 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 -- 172.21.15.129:0/2369124209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 msgr2=0x7f34e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 --2- 172.21.15.129:0/2369124209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e4106e60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009930 tx=0x7f34dc02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 -- 172.21.15.129:0/2369124209 shutdown_connections 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 --2- 172.21.15.129:0/2369124209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4101420 0x7f34e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 --2- 172.21.15.129:0/2369124209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34e4100a80 0x7f34e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 --2- 172.21.15.129:0/2369124209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e4106e60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 -- 172.21.15.129:0/2369124209 >> 172.21.15.129:0/2369124209 conn(0x7f34e40fc7a0 msgr2=0x7f34e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 -- 172.21.15.129:0/2369124209 shutdown_connections 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 -- 172.21.15.129:0/2369124209 wait complete. 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.669+0000 7f34eb52a640 1 Processor -- start 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 -- start start 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4100a80 0x7f34e40fce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34e4101420 0x7f34e40fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e40fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4110d30 con 0x7f34e4106a80 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4110bb0 con 0x7f34e4100a80 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34eb52a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f34e4110eb0 con 0x7f34e4101420 2024-09-05T01:14:27.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e40fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e40fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48668/0 (socket says 172.21.15.129:48668) 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 -- 172.21.15.129:0/1466782292 learned_addr learned my addr 172.21.15.129:0/1466782292 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e92c6640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4100a80 0x7f34e40fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 -- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34e4101420 msgr2=0x7f34e40fd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f34e4101420 0x7f34e40fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 -- 172.21.15.129:0/1466782292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4100a80 msgr2=0x7f34e40fce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4100a80 0x7f34e40fce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e41a6860 con 0x7f34e4106a80 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e92c6640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f34e4100a80 0x7f34e40fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e9ac7640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e40fd900 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f34d400eb10 tx=0x7f34d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.670+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4004b20 con 0x7f34e4106a80 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.671+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34d4004cc0 con 0x7f34e4106a80 2024-09-05T01:14:27.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.671+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e41a6b50 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.671+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4010690 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.671+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e41a7040 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.671+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f34e40628c0 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.673+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f34d4005000 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.673+0000 7f34e27fc640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f34cc0a2c10 0x7f34cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.673+0000 7f34e92c6640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f34cc0a2c10 0x7f34cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.673+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f34d40d0010 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e27fc640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] conn(0x7f34cc0add60 0x7f34cc0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f34d40047f0 con 0x7f34cc0add60 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f34d40d0460 con 0x7f34e4106a80 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e92c6640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f34cc0a2c10 0x7f34cc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e8ac5640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] conn(0x7f34cc0add60 0x7f34cc0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.674+0000 7f34e8ac5640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] conn(0x7f34cc0add60 0x7f34cc0b0200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:27.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.675+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== osd.8 v2:172.21.15.194:6816/307349616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f34d40047f0 con 0x7f34cc0add60 2024-09-05T01:14:27.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.689+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 --> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f34e4062d40 con 0x7f34cc0add60 2024-09-05T01:14:27.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.689+0000 7f34e27fc640 1 -- 172.21.15.129:0/1466782292 <== osd.8 v2:172.21.15.194:6816/307349616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f34e4062d40 con 0x7f34cc0add60 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.689+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] conn(0x7f34cc0add60 msgr2=0x7f34cc0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.689+0000 7f34eb52a640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] conn(0x7f34cc0add60 0x7f34cc0b0200 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f34cc0a2c10 msgr2=0x7f34cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f34cc0a2c10 0x7f34cc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f34dc000c00 tx=0x7f34dc009610 comp rx=0 tx=0).stop 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 msgr2=0x7f34e40fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 --2- 172.21.15.129:0/1466782292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f34e4106a80 0x7f34e40fd900 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f34d400eb10 tx=0x7f34d400c2d0 comp rx=0 tx=0).stop 2024-09-05T01:14:27.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34e9ac7640 1 -- 172.21.15.129:0/1466782292 reap_dead start 2024-09-05T01:14:27.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 shutdown_connections 2024-09-05T01:14:27.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 >> 172.21.15.129:0/1466782292 conn(0x7f34e40fc7a0 msgr2=0x7f34e406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:27.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 shutdown_connections 2024-09-05T01:14:27.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:27.690+0000 7f34eb52a640 1 -- 172.21.15.129:0/1466782292 wait complete. 2024-09-05T01:14:28.164 INFO:teuthology.orchestra.run.smithi129.stdout:103079215173 2024-09-05T01:14:28.164 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-05T01:14:28.815 INFO:teuthology.orchestra.run.smithi129.stdout:236223201325 2024-09-05T01:14:28.816 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-05T01:14:28.822 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:28.822 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:28.881 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:29.057 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:28 smithi129 ceph-mon[30979]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-05T01:14:29.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:28 smithi183 ceph-mon[34589]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-05T01:14:29.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:28 smithi194 ceph-mon[34941]: pgmap v323: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 -- 172.21.15.129:0/2785509889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 msgr2=0x7f1a4410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/2785509889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a4410d950 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009cd0 tx=0x7f1a3c02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 -- 172.21.15.129:0/2785509889 shutdown_connections 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/2785509889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a4410d950 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/2785509889 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a44100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/2785509889 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a44106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 -- 172.21.15.129:0/2785509889 >> 172.21.15.129:0/2785509889 conn(0x7f1a440fc7a0 msgr2=0x7f1a440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 -- 172.21.15.129:0/2785509889 shutdown_connections 2024-09-05T01:14:29.599 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.587+0000 7f1a4ba55640 1 -- 172.21.15.129:0/2785509889 wait complete. 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 Processor -- start 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 -- start start 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a441971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a4419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44110d30 con 0x7f1a44101420 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44110bb0 con 0x7f1a44100a80 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a4ba55640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44110eb0 con 0x7f1a44106a80 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a48ff0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a48ff0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48684/0 (socket says 172.21.15.129:48684) 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.588+0000 7f1a48ff0640 1 -- 172.21.15.129:0/3911032466 learned_addr learned my addr 172.21.15.129:0/3911032466 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:29.600 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a497f1640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a441971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a49ff2640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a4419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 -- 172.21.15.129:0/3911032466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 msgr2=0x7f1a4419baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a4419baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 -- 172.21.15.129:0/3911032466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 msgr2=0x7f1a441971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a441971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 -- 172.21.15.129:0/3911032466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4419c180 con 0x7f1a44101420 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a49ff2640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a4419baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a497f1640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a441971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a48ff0640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c920 tx=0x7f1a3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34007c70 con 0x7f1a44101420 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4419c470 con 0x7f1a44101420 2024-09-05T01:14:29.601 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a34007e10 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34018820 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.589+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a3d50 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.590+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44100ee0 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.591+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1a340189c0 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.592+0000 7f1a3a7fc640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 0x7f1a300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.592+0000 7f1a497f1640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 0x7f1a300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.592+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1a340cf750 con 0x7f1a44101420 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.592+0000 7f1a497f1640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 0x7f1a300a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005e00 tx=0x7f1a40005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:29.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.596+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a34014030 con 0x7f1a44101420 2024-09-05T01:14:29.626 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:29.626 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:29.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.772+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1a44108c40 con 0x7f1a44101420 2024-09-05T01:14:29.773 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.772+0000 7f1a3a7fc640 1 -- 172.21.15.129:0/3911032466 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1a3409c1f0 con 0x7f1a44101420 2024-09-05T01:14:29.773 INFO:teuthology.orchestra.run.smithi129.stdout:184683593783 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.774+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 msgr2=0x7f1a300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.774+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 0x7f1a300a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1a40005e00 tx=0x7f1a40005c70 comp rx=0 tx=0).stop 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.774+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 msgr2=0x7f1a44197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.774+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c920 tx=0x7f1a3400cde0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.774+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 shutdown_connections 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a300a2c10 0x7f1a300a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a44106a80 0x7f1a4419baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a44101420 0x7f1a44197710 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 --2- 172.21.15.129:0/3911032466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a44100a80 0x7f1a441971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:29.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 >> 172.21.15.129:0/3911032466 conn(0x7f1a440fc7a0 msgr2=0x7f1a440fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:29.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 shutdown_connections 2024-09-05T01:14:29.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:29.775+0000 7f1a4ba55640 1 -- 172.21.15.129:0/3911032466 wait complete. 2024-09-05T01:14:29.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:29 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3911032466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T01:14:30.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:29 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3911032466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 -- 172.21.15.129:0/1070415334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48106a80 msgr2=0x7fac48106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/1070415334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48106a80 0x7fac48106e60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac44009930 tx=0x7fac4402a420 comp rx=0 tx=0).stop 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 -- 172.21.15.129:0/1070415334 shutdown_connections 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/1070415334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48101420 0x7fac4810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/1070415334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48100a80 0x7fac48100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/1070415334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48106a80 0x7fac48106e60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 -- 172.21.15.129:0/1070415334 >> 172.21.15.129:0/1070415334 conn(0x7fac480fc7a0 msgr2=0x7fac480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 -- 172.21.15.129:0/1070415334 shutdown_connections 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.258+0000 7fac4f8c0640 1 -- 172.21.15.129:0/1070415334 wait complete. 2024-09-05T01:14:30.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 Processor -- start 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 -- start start 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48100a80 0x7fac480fcf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 0x7fac480fd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fac48110dd0 con 0x7fac48101420 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fac48110c50 con 0x7fac48100a80 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4f8c0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fac48110f50 con 0x7fac48106a80 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4ce5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4ce5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48706/0 (socket says 172.21.15.129:48706) 2024-09-05T01:14:30.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.259+0000 7fac4ce5b640 1 -- 172.21.15.129:0/3401397804 learned_addr learned my addr 172.21.15.129:0/3401397804 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4de5d640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 0x7fac480fd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4d65c640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48100a80 0x7fac480fcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 -- 172.21.15.129:0/3401397804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 msgr2=0x7fac480fd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 0x7fac480fd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 -- 172.21.15.129:0/3401397804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48100a80 msgr2=0x7fac480fcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48100a80 0x7fac480fcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 -- 172.21.15.129:0/3401397804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac481a6980 con 0x7fac48101420 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4de5d640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 0x7fac480fd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4ce5b640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac3800b4a0 tx=0x7fac3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38014070 con 0x7fac48101420 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac38010ac0 con 0x7fac48101420 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38017c30 con 0x7fac48101420 2024-09-05T01:14:30.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481a6c70 con 0x7fac48101420 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.260+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a71b0 con 0x7fac48101420 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.261+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac48100f60 con 0x7fac48101420 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.262+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fac38017070 con 0x7fac48101420 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.263+0000 7fac3e7fc640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 0x7fac340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.263+0000 7fac4d65c640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 0x7fac340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.263+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fac380ceba0 con 0x7fac48101420 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.264+0000 7fac4d65c640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 0x7fac340a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac44031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:30.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.267+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac3809b750 con 0x7fac48101420 2024-09-05T01:14:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:29 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3911032466' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T01:14:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.442+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fac48109a70 con 0x7fac48101420 2024-09-05T01:14:30.443 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.443+0000 7fac3e7fc640 1 -- 172.21.15.129:0/3401397804 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fac3809b050 con 0x7fac48101420 2024-09-05T01:14:30.443 INFO:teuthology.orchestra.run.smithi129.stdout:154618822716 2024-09-05T01:14:30.445 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 msgr2=0x7fac340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 0x7fac340a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac44031400 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 msgr2=0x7fac480fd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac3800b4a0 tx=0x7fac3800b960 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 shutdown_connections 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fac340a2c10 0x7fac340a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fac48106a80 0x7fac480fd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fac48101420 0x7fac480fd480 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 --2- 172.21.15.129:0/3401397804 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fac48100a80 0x7fac480fcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 >> 172.21.15.129:0/3401397804 conn(0x7fac480fc7a0 msgr2=0x7fac4806ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 shutdown_connections 2024-09-05T01:14:30.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.445+0000 7fac4f8c0640 1 -- 172.21.15.129:0/3401397804 wait complete. 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- 172.21.15.129:0/3195599535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a010c170 msgr2=0x7f67a010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/3195599535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a010c170 0x7f67a010e610 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6798009ca0 tx=0x7f679802a950 comp rx=0 tx=0).stop 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- 172.21.15.129:0/3195599535 shutdown_connections 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/3195599535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a010c170 0x7f67a010e610 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/3195599535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 0x7f67a010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/3195599535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a0102d40 0x7f67a0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- 172.21.15.129:0/3195599535 >> 172.21.15.129:0/3195599535 conn(0x7f67a00fc7a0 msgr2=0x7f67a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- 172.21.15.129:0/3195599535 shutdown_connections 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- 172.21.15.129:0/3195599535 wait complete. 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 Processor -- start 2024-09-05T01:14:30.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.799+0000 7f67a5d1d640 1 -- start start 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 0x7f67a0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 0x7f67a0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0111e00 con 0x7f67a0102d40 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0111c80 con 0x7f67a010c170 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67a0111f80 con 0x7f67a0103660 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48730/0 (socket says 172.21.15.129:48730) 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 -- 172.21.15.129:0/4114350026 learned_addr learned my addr 172.21.15.129:0/4114350026 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f6796ffd640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 0x7f67a0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679ffff640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 0x7f67a0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 -- 172.21.15.129:0/4114350026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 msgr2=0x7f67a0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 0x7f67a0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 -- 172.21.15.129:0/4114350026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 msgr2=0x7f67a0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 0x7f67a0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 -- 172.21.15.129:0/4114350026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a01a6870 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679ffff640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 0x7f67a0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679f7fe640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f679000e9f0 tx=0x7f679000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679000cd80 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f67900108e0 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a01a6b60 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.800+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6790018920 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.801+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a01a70a0 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.802+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67a0103120 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.803+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f6790010a90 con 0x7f67a0102d40 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.803+0000 7f679d7fa640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 0x7f678c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:30.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.804+0000 7f6796ffd640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 0x7f678c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:30.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.804+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f67900cf8b0 con 0x7f67a0102d40 2024-09-05T01:14:30.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.804+0000 7f6796ffd640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 0x7f678c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67a010d5d0 tx=0x7f678800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:30.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.807+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f679009c350 con 0x7f67a0102d40 2024-09-05T01:14:30.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.977+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f67a01071e0 con 0x7f67a0102d40 2024-09-05T01:14:30.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:30 smithi129 ceph-mon[30979]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-05T01:14:30.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3401397804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T01:14:30.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.978+0000 7f679d7fa640 1 -- 172.21.15.129:0/4114350026 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f679009bc50 con 0x7f67a0102d40 2024-09-05T01:14:30.979 INFO:teuthology.orchestra.run.smithi129.stdout:266287972392 2024-09-05T01:14:30.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 msgr2=0x7f678c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 0x7f678c0a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f67a010d5d0 tx=0x7f678800b2e0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 msgr2=0x7f67a01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:30.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f679000e9f0 tx=0x7f679000eeb0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 shutdown_connections 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f678c0a2c10 0x7f678c0a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f67a010c170 0x7f67a0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f67a0103660 0x7f67a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 --2- 172.21.15.129:0/4114350026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67a0102d40 0x7f67a01009a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 >> 172.21.15.129:0/4114350026 conn(0x7f67a00fc7a0 msgr2=0x7f67a00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 shutdown_connections 2024-09-05T01:14:30.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:30.981+0000 7f67a5d1d640 1 -- 172.21.15.129:0/4114350026 wait complete. 2024-09-05T01:14:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:30 smithi194 ceph-mon[34941]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-05T01:14:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3401397804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T01:14:31.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:30 smithi183 ceph-mon[34589]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-05T01:14:31.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3401397804' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T01:14:31.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972390 got 266287972392 for osd.9 2024-09-05T01:14:31.724 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:31.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:31 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4114350026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T01:14:31.986 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:31 smithi129 ceph-mon[30979]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 -- 172.21.15.129:0/1926456796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 msgr2=0x7f1dfc10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 --2- 172.21.15.129:0/1926456796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc10eac0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009ca0 tx=0x7f1df402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 -- 172.21.15.129:0/1926456796 shutdown_connections 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 --2- 172.21.15.129:0/1926456796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc10eac0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 --2- 172.21.15.129:0/1926456796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc1027c0 0x7f1dfc106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.009 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 --2- 172.21.15.129:0/1926456796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc101e10 0x7f1dfc1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 -- 172.21.15.129:0/1926456796 >> 172.21.15.129:0/1926456796 conn(0x7f1dfc0fd8d0 msgr2=0x7f1dfc0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 -- 172.21.15.129:0/1926456796 shutdown_connections 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.985+0000 7f1e02107640 1 -- 172.21.15.129:0/1926456796 wait complete. 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.986+0000 7f1e02107640 1 Processor -- start 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.986+0000 7f1e02107640 1 -- start start 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101e10 0x7f1dfc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc1027c0 0x7f1dfc198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc111e50 con 0x7f1dfc106f40 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc111cd0 con 0x7f1dfc101e10 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1e02107640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1dfc111fd0 con 0x7f1dfc1027c0 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48746/0 (socket says 172.21.15.129:48746) 2024-09-05T01:14:32.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 -- 172.21.15.129:0/3179002039 learned_addr learned my addr 172.21.15.129:0/3179002039 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfaffd640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc1027c0 0x7f1dfc198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfb7fe640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101e10 0x7f1dfc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 -- 172.21.15.129:0/3179002039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc1027c0 msgr2=0x7f1dfc198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc1027c0 0x7f1dfc198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 -- 172.21.15.129:0/3179002039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101e10 msgr2=0x7f1dfc1982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101e10 0x7f1dfc1982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 -- 172.21.15.129:0/3179002039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dfc19d290 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.987+0000 7f1dfbfff640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009dc0 tx=0x7f1df4032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.988+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4003cf0 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.988+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dfc19d520 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.988+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df40044f0 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.988+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df4030bb0 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.988+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dfc1a4e60 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.989+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dfc1021f0 con 0x7f1dfc106f40 2024-09-05T01:14:32.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.990+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1df4004040 con 0x7f1dfc106f40 2024-09-05T01:14:32.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.991+0000 7f1df8ff9640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 0x7f1de40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.991+0000 7f1dfb7fe640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 0x7f1de40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.991+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1df40f0c40 con 0x7f1dfc106f40 2024-09-05T01:14:32.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.991+0000 7f1dfb7fe640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 0x7f1de40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005d00 tx=0x7f1dec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:32.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:31.994+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1df40bd910 con 0x7f1dfc106f40 2024-09-05T01:14:32.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.165+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1dfc199640 con 0x7f1dfc106f40 2024-09-05T01:14:32.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.166+0000 7f1df8ff9640 1 -- 172.21.15.129:0/3179002039 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1dfc199640 con 0x7f1dfc106f40 2024-09-05T01:14:32.167 INFO:teuthology.orchestra.run.smithi129.stdout:206158430259 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 msgr2=0x7f1de40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 0x7f1de40a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005d00 tx=0x7f1dec00b410 comp rx=0 tx=0).stop 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 msgr2=0x7f1dfc19cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009dc0 tx=0x7f1df4032ff0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 shutdown_connections 2024-09-05T01:14:32.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2c10 0x7f1de40a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dfc106f40 0x7f1dfc19cbb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1dfc1027c0 0x7f1dfc198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 --2- 172.21.15.129:0/3179002039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1dfc101e10 0x7f1dfc1982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 >> 172.21.15.129:0/3179002039 conn(0x7f1dfc0fd8d0 msgr2=0x7f1dfc0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 shutdown_connections 2024-09-05T01:14:32.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.168+0000 7f1e02107640 1 -- 172.21.15.129:0/3179002039 wait complete. 2024-09-05T01:14:32.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- 172.21.15.129:0/1501170186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274100100 msgr2=0x7f6274100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 --2- 172.21.15.129:0/1501170186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274100100 0x7f6274100560 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f625c009900 tx=0x7f625c02a410 comp rx=0 tx=0).stop 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- 172.21.15.129:0/1501170186 shutdown_connections 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 --2- 172.21.15.129:0/1501170186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100aa0 0x7f627410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 --2- 172.21.15.129:0/1501170186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274100100 0x7f6274100560 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 --2- 172.21.15.129:0/1501170186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274106b80 0x7f6274106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- 172.21.15.129:0/1501170186 >> 172.21.15.129:0/1501170186 conn(0x7f6274077230 msgr2=0x7f62740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:32.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- 172.21.15.129:0/1501170186 shutdown_connections 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- 172.21.15.129:0/1501170186 wait complete. 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 Processor -- start 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.200+0000 7f627a8d6640 1 -- start start 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100100 0x7f6274198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274100aa0 0x7f6274198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6274111ec0 con 0x7f6274106b80 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6274111d40 con 0x7f6274100aa0 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6274112040 con 0x7f6274100100 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55188/0 (socket says 172.21.15.129:55188) 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 -- 172.21.15.129:0/513761359 learned_addr learned my addr 172.21.15.129:0/513761359 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:32.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f626bfff640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274100aa0 0x7f6274198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6273fff640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100100 0x7f6274198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 -- 172.21.15.129:0/513761359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100100 msgr2=0x7f6274198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100100 0x7f6274198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 -- 172.21.15.129:0/513761359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274100aa0 msgr2=0x7f6274198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274100aa0 0x7f6274198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 -- 172.21.15.129:0/513761359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627419d400 con 0x7f6274106b80 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6278e73640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f626c00eb40 tx=0x7f626c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:32.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c004b20 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627419d6f0 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f626c004cc0 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f626c0107b0 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.201+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62741a4fd0 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.202+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6274100560 con 0x7f6274106b80 2024-09-05T01:14:32.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.203+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f626c005020 con 0x7f6274106b80 2024-09-05T01:14:32.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.204+0000 7f6271ffb640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 0x7f62600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:32.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.204+0000 7f6273fff640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 0x7f62600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:32.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.204+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f626c0cfdc0 con 0x7f6274106b80 2024-09-05T01:14:32.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.205+0000 7f6273fff640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 0x7f62600a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6264005fd0 tx=0x7f6264005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:32.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.208+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f626c09c800 con 0x7f6274106b80 2024-09-05T01:14:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:31 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4114350026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T01:14:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:31 smithi194 ceph-mon[34941]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.381+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f627410a0e0 con 0x7f6274106b80 2024-09-05T01:14:32.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.382+0000 7f6271ffb640 1 -- 172.21.15.129:0/513761359 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f626c09c100 con 0x7f6274106b80 2024-09-05T01:14:32.382 INFO:teuthology.orchestra.run.smithi129.stdout:287762808867 2024-09-05T01:14:32.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 msgr2=0x7f62600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 0x7f62600a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6264005fd0 tx=0x7f6264005850 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 msgr2=0x7f627419cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f626c00eb40 tx=0x7f626c00c2d0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 shutdown_connections 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62600a2c10 0x7f62600a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6274106b80 0x7f627419cd20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6274100aa0 0x7f6274198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 --2- 172.21.15.129:0/513761359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6274100100 0x7f6274198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 >> 172.21.15.129:0/513761359 conn(0x7f6274077230 msgr2=0x7f6274104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 shutdown_connections 2024-09-05T01:14:32.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:32.384+0000 7f627a8d6640 1 -- 172.21.15.129:0/513761359 wait complete. 2024-09-05T01:14:32.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:31 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4114350026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T01:14:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:31 smithi183 ceph-mon[34589]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:32.460 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:32.461 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:32.876 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430256 got 206158430259 for osd.7 2024-09-05T01:14:32.876 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:32.888 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:33.188 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:32 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3179002039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T01:14:33.188 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:32 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/513761359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T01:14:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:32 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3179002039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T01:14:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:32 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/513761359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T01:14:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:32 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3179002039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T01:14:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:32 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/513761359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T01:14:33.488 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822713 got 154618822716 for osd.5 2024-09-05T01:14:33.489 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:33 smithi129 ceph-mon[30979]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:34.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:33 smithi194 ceph-mon[34941]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:34.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:33 smithi183 ceph-mon[34589]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.459+0000 7f08b6884640 1 -- 172.21.15.129:0/952961462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0101de0 msgr2=0x7f08b01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.459+0000 7f08b6884640 1 --2- 172.21.15.129:0/952961462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0101de0 0x7f08b01021c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009900 tx=0x7f08a002a410 comp rx=0 tx=0).stop 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- 172.21.15.129:0/952961462 shutdown_connections 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- 172.21.15.129:0/952961462 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0106f10 0x7f08b010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- 172.21.15.129:0/952961462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0102790 0x7f08b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- 172.21.15.129:0/952961462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0101de0 0x7f08b01021c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- 172.21.15.129:0/952961462 >> 172.21.15.129:0/952961462 conn(0x7f08b00fd8c0 msgr2=0x7f08b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- 172.21.15.129:0/952961462 shutdown_connections 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- 172.21.15.129:0/952961462 wait complete. 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 Processor -- start 2024-09-05T01:14:34.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- start start 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0101de0 0x7f08b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0106f10 0x7f08b019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0111e20 con 0x7f08b0102790 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0111ca0 con 0x7f08b0106f10 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.460+0000 7f08b6884640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0111fa0 con 0x7f08b0101de0 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55202/0 (socket says 172.21.15.129:55202) 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 -- 172.21.15.129:0/2051889288 learned_addr learned my addr 172.21.15.129:0/2051889288 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08affff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0101de0 0x7f08b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08b4e21640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0106f10 0x7f08b019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 -- 172.21.15.129:0/2051889288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0101de0 msgr2=0x7f08b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0101de0 0x7f08b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 -- 172.21.15.129:0/2051889288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0106f10 msgr2=0x7f08b019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0106f10 0x7f08b019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 -- 172.21.15.129:0/2051889288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b019d260 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08a7fff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f089800c920 tx=0x7f089800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898007c70 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0898007e10 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b019d550 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898018820 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.461+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a4e30 con 0x7f08b0102790 2024-09-05T01:14:34.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.462+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b01021c0 con 0x7f08b0102790 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.463+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f08980189c0 con 0x7f08b0102790 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.464+0000 7f08adffb640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 0x7f089c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.464+0000 7f08affff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 0x7f089c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.464+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f08980cf600 con 0x7f08b0102790 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.465+0000 7f08affff640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 0x7f089c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08a0031440 tx=0x7f08a0005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:34.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.468+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0898014030 con 0x7f08b0102790 2024-09-05T01:14:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.638+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f08b0109b60 con 0x7f08b0102790 2024-09-05T01:14:34.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.639+0000 7f08adffb640 1 -- 172.21.15.129:0/2051889288 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f089809c0a0 con 0x7f08b0102790 2024-09-05T01:14:34.639 INFO:teuthology.orchestra.run.smithi129.stdout:55834574929 2024-09-05T01:14:34.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 msgr2=0x7f089c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 0x7f089c0a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08a0031440 tx=0x7f08a0005e80 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 msgr2=0x7f08b01987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f089800c920 tx=0x7f089800cde0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 shutdown_connections 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f089c0a2c90 0x7f089c0a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f08b0106f10 0x7f08b019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0102790 0x7f08b01987f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 --2- 172.21.15.129:0/2051889288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f08b0101de0 0x7f08b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 >> 172.21.15.129:0/2051889288 conn(0x7f08b00fd8c0 msgr2=0x7f08b00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 shutdown_connections 2024-09-05T01:14:34.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:34.641+0000 7f08b6884640 1 -- 172.21.15.129:0/2051889288 wait complete. 2024-09-05T01:14:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:34.932 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808867 for osd.10 2024-09-05T01:14:34.932 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:34 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2051889288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T01:14:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:34 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2051889288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T01:14:35.348 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593781 got 184683593783 for osd.6 2024-09-05T01:14:35.348 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:35.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:34 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2051889288' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.551+0000 7f21703b6640 1 -- 172.21.15.129:0/3357852318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168103780 msgr2=0x7f2168107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.551+0000 7f21703b6640 1 --2- 172.21.15.129:0/3357852318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168103780 0x7f2168107bc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f215c009930 tx=0x7f215c02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 -- 172.21.15.129:0/3357852318 shutdown_connections 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 --2- 172.21.15.129:0/3357852318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168108300 0x7f216810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 --2- 172.21.15.129:0/3357852318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168103780 0x7f2168107bc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 --2- 172.21.15.129:0/3357852318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168102dd0 0x7f21681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 -- 172.21.15.129:0/3357852318 >> 172.21.15.129:0/3357852318 conn(0x7f21680fc7d0 msgr2=0x7f21680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 -- 172.21.15.129:0/3357852318 shutdown_connections 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 -- 172.21.15.129:0/3357852318 wait complete. 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 Processor -- start 2024-09-05T01:14:35.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.552+0000 7f21703b6640 1 -- start start 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 0x7f2168198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168108300 0x7f216819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2168111f00 con 0x7f2168102dd0 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2168111d80 con 0x7f2168108300 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21703b6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2168112080 con 0x7f2168103780 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216d951640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 0x7f2168198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216d951640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:56820/0 (socket says 172.21.15.129:56820) 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216d951640 1 -- 172.21.15.129:0/1234014852 learned_addr learned my addr 172.21.15.129:0/1234014852 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e953640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168108300 0x7f216819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 -- 172.21.15.129:0/1234014852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 msgr2=0x7f2168198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:35.576 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 -- 172.21.15.129:0/1234014852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168108300 msgr2=0x7f216819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168108300 0x7f216819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 -- 172.21.15.129:0/1234014852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216819d4b0 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216d951640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f216e152640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 0x7f2168198500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f215800cce0 tx=0x7f21580075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.553+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2158004b20 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.554+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2158004cc0 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.554+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216819d7a0 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.554+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800f6b0 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.554+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a5080 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.554+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21681031b0 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.556+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2158005000 con 0x7f2168102dd0 2024-09-05T01:14:35.577 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.556+0000 7f21677fe640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 0x7f21500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:35.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.557+0000 7f216d951640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 0x7f21500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:35.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.557+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f21580cf770 con 0x7f2168102dd0 2024-09-05T01:14:35.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.557+0000 7f216d951640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 0x7f21500a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f215c005ec0 tx=0x7f215c0316e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:35.578 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.560+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2158014030 con 0x7f2168102dd0 2024-09-05T01:14:35.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.732+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f2168199860 con 0x7f2168102dd0 2024-09-05T01:14:35.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.732+0000 7f21677fe640 1 -- 172.21.15.129:0/1234014852 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f2168199860 con 0x7f2168102dd0 2024-09-05T01:14:35.733 INFO:teuthology.orchestra.run.smithi129.stdout:317827579935 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 msgr2=0x7f21500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 0x7f21500a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f215c005ec0 tx=0x7f215c0316e0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 msgr2=0x7f2168198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 0x7f2168198500 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f215800cce0 tx=0x7f21580075f0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 shutdown_connections 2024-09-05T01:14:35.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f21500a2c90 0x7f21500a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2168108300 0x7f216819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2168103780 0x7f2168198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.734+0000 7f21703b6640 1 --2- 172.21.15.129:0/1234014852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2168102dd0 0x7f2168198500 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.735+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 >> 172.21.15.129:0/1234014852 conn(0x7f21680fc7d0 msgr2=0x7f2168105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.735+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 shutdown_connections 2024-09-05T01:14:35.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:35.735+0000 7f21703b6640 1 -- 172.21.15.129:0/1234014852 wait complete. 2024-09-05T01:14:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:35 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1234014852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T01:14:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:35 smithi129 ceph-mon[30979]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-05T01:14:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:35 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1234014852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T01:14:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:35 smithi194 ceph-mon[34941]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-05T01:14:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:35 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1234014852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T01:14:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:35 smithi183 ceph-mon[34589]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-05T01:14:36.413 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579934 got 317827579935 for osd.11 2024-09-05T01:14:36.414 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:37.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574929 for osd.1 2024-09-05T01:14:37.240 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 -- 172.21.15.129:0/536443538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c106a80 msgr2=0x7f5f0c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 --2- 172.21.15.129:0/536443538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c106a80 0x7f5f0c106e60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009930 tx=0x7f5efc02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 -- 172.21.15.129:0/536443538 shutdown_connections 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 --2- 172.21.15.129:0/536443538 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c101420 0x7f5f0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 --2- 172.21.15.129:0/536443538 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 0x7f5f0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 --2- 172.21.15.129:0/536443538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c106a80 0x7f5f0c106e60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 -- 172.21.15.129:0/536443538 >> 172.21.15.129:0/536443538 conn(0x7f5f0c0fc7a0 msgr2=0x7f5f0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 -- 172.21.15.129:0/536443538 shutdown_connections 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 -- 172.21.15.129:0/536443538 wait complete. 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.452+0000 7f5f11cca640 1 Processor -- start 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 -- start start 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 0x7f5f0c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 0x7f5f0c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c110cf0 con 0x7f5f0c101420 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c110b70 con 0x7f5f0c100a80 2024-09-05T01:14:37.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f11cca640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5f0c110e70 con 0x7f5f0c106a80 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55226/0 (socket says 172.21.15.129:55226) 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 -- 172.21.15.129:0/3370444780 learned_addr learned my addr 172.21.15.129:0/3370444780 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0b7fe640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 0x7f5f0c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0bfff640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 0x7f5f0c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 -- 172.21.15.129:0/3370444780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 msgr2=0x7f5f0c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 0x7f5f0c1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 -- 172.21.15.129:0/3370444780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 msgr2=0x7f5f0c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 0x7f5f0c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0affd640 1 -- 172.21.15.129:0/3370444780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0c1a47c0 con 0x7f5f0c101420 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.453+0000 7f5f0bfff640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 0x7f5f0c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f0affd640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000d950 tx=0x7f5f0000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:37.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00017070 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0c1a4990 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5f00009c30 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000f620 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.454+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0c1a4ed0 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.455+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f0c100ee0 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.456+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5f000109a0 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.457+0000 7f5f08ff9640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.457+0000 7f5f0b7fe640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.457+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f5f000cf140 con 0x7f5f0c101420 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.457+0000 7f5f0b7fe640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009900 tx=0x7f5efc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:37.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.460+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f00009de0 con 0x7f5f0c101420 2024-09-05T01:14:37.471 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:37.471 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:37.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.636+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5f0c108ff0 con 0x7f5f0c101420 2024-09-05T01:14:37.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.637+0000 7f5f08ff9640 1 -- 172.21.15.129:0/3370444780 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5f0009bbe0 con 0x7f5f0c101420 2024-09-05T01:14:37.638 INFO:teuthology.orchestra.run.smithi129.stdout:34359738454 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 msgr2=0x7f5ef40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009900 tx=0x7f5efc000bc0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 msgr2=0x7f5f0c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000d950 tx=0x7f5f0000de10 comp rx=0 tx=0).stop 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 shutdown_connections 2024-09-05T01:14:37.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ef40a2c10 0x7f5ef40a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5f0c106a80 0x7f5f0c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5f0c101420 0x7f5f0c199740 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 --2- 172.21.15.129:0/3370444780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5f0c100a80 0x7f5f0c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 >> 172.21.15.129:0/3370444780 conn(0x7f5f0c0fc7a0 msgr2=0x7f5f0c0fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 shutdown_connections 2024-09-05T01:14:37.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:37.639+0000 7f5f11cca640 1 -- 172.21.15.129:0/3370444780 wait complete. 2024-09-05T01:14:37.728 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:37 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3370444780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T01:14:38.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:37 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3370444780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T01:14:38.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 -- 172.21.15.129:0/863341796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8101de0 msgr2=0x7f25e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 --2- 172.21.15.129:0/863341796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8101de0 0x7f25e81021c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009900 tx=0x7f25d802a410 comp rx=0 tx=0).stop 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 -- 172.21.15.129:0/863341796 shutdown_connections 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 --2- 172.21.15.129:0/863341796 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8106f10 0x7f25e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 --2- 172.21.15.129:0/863341796 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8102790 0x7f25e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 --2- 172.21.15.129:0/863341796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8101de0 0x7f25e81021c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 -- 172.21.15.129:0/863341796 >> 172.21.15.129:0/863341796 conn(0x7f25e80fd8c0 msgr2=0x7f25e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 -- 172.21.15.129:0/863341796 shutdown_connections 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.115+0000 7f25ede45640 1 -- 172.21.15.129:0/863341796 wait complete. 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 Processor -- start 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 -- start start 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8101de0 0x7f25e8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.129 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8102790 0x7f25e81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8111e20 con 0x7f25e8106f10 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8111ca0 con 0x7f25e8102790 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25ede45640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25e8111fa0 con 0x7f25e8101de0 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55230/0 (socket says 172.21.15.129:55230) 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25e7fff640 1 -- 172.21.15.129:0/805232123 learned_addr learned my addr 172.21.15.129:0/805232123 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25e77fe640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8101de0 0x7f25e8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.116+0000 7f25e6ffd640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8102790 0x7f25e81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 -- 172.21.15.129:0/805232123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8101de0 msgr2=0x7f25e8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8101de0 0x7f25e8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 -- 172.21.15.129:0/805232123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8102790 msgr2=0x7f25e81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.130 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8102790 0x7f25e81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 -- 172.21.15.129:0/805232123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e819d330 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e7fff640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f25e000eaa0 tx=0x7f25e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e000cdb0 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25e0005020 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e819d620 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e00116d0 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.117+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81a4f00 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.118+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e81021c0 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.119+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f25e00051d0 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.120+0000 7f25e4ff9640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 0x7f25d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.120+0000 7f25e77fe640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 0x7f25d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.120+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f25e00cf790 con 0x7f25e8106f10 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.120+0000 7f25e77fe640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 0x7f25d40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031440 tx=0x7f25d80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:38.131 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.124+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25e0016030 con 0x7f25e8106f10 2024-09-05T01:14:38.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:37 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3370444780' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T01:14:38.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.297+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f25e8109b60 con 0x7f25e8106f10 2024-09-05T01:14:38.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.298+0000 7f25e4ff9640 1 -- 172.21.15.129:0/805232123 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f25e009c230 con 0x7f25e8106f10 2024-09-05T01:14:38.298 INFO:teuthology.orchestra.run.smithi129.stdout:77309411405 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 msgr2=0x7f25d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 0x7f25d40a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25d8031440 tx=0x7f25d80095f0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 msgr2=0x7f25e819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f25e000eaa0 tx=0x7f25e000ef60 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 shutdown_connections 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25d40a2c10 0x7f25d40a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25e8106f10 0x7f25e819cc50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25e8102790 0x7f25e81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 --2- 172.21.15.129:0/805232123 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f25e8101de0 0x7f25e8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 >> 172.21.15.129:0/805232123 conn(0x7f25e80fd8c0 msgr2=0x7f25e80fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:38.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 shutdown_connections 2024-09-05T01:14:38.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.300+0000 7f25ede45640 1 -- 172.21.15.129:0/805232123 wait complete. 2024-09-05T01:14:38.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 -- 172.21.15.129:0/3466521091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 msgr2=0x7face0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 --2- 172.21.15.129:0/3466521091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face0106e60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7facd0009930 tx=0x7facd002a420 comp rx=0 tx=0).stop 2024-09-05T01:14:38.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 -- 172.21.15.129:0/3466521091 shutdown_connections 2024-09-05T01:14:38.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 --2- 172.21.15.129:0/3466521091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0101420 0x7face010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 --2- 172.21.15.129:0/3466521091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0100a80 0x7face0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 --2- 172.21.15.129:0/3466521091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face0106e60 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 -- 172.21.15.129:0/3466521091 >> 172.21.15.129:0/3466521091 conn(0x7face00fc7a0 msgr2=0x7face00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 -- 172.21.15.129:0/3466521091 shutdown_connections 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.663+0000 7face5892640 1 -- 172.21.15.129:0/3466521091 wait complete. 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 Processor -- start 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 -- start start 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0100a80 0x7face01991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0101420 0x7face0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7face0110d30 con 0x7face0106a80 2024-09-05T01:14:38.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7face0110bb0 con 0x7face0101420 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7face5892640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7face0110eb0 con 0x7face0100a80 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7facdf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7facdf7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55258/0 (socket says 172.21.15.129:55258) 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.664+0000 7facdf7fe640 1 -- 172.21.15.129:0/2898506912 learned_addr learned my addr 172.21.15.129:0/2898506912 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facde7fc640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0101420 0x7face0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdeffd640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0100a80 0x7face01991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 -- 172.21.15.129:0/2898506912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0100a80 msgr2=0x7face01991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0100a80 0x7face01991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 -- 172.21.15.129:0/2898506912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0101420 msgr2=0x7face0199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0101420 0x7face0199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 -- 172.21.15.129:0/2898506912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face01a47a0 con 0x7face0106a80 2024-09-05T01:14:38.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7facdf7fe640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facd800eaa0 tx=0x7facd800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800cdb0 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facd8005020 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face01a4970 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80116d0 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.665+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face01a4eb0 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.666+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face0100ee0 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.667+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7facd80051d0 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.668+0000 7face4890640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 0x7facc80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.668+0000 7facdeffd640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 0x7facc80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.668+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7facd80cf520 con 0x7face0106a80 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.669+0000 7facdeffd640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 0x7facc80a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:38.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.673+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facd8016030 con 0x7face0106a80 2024-09-05T01:14:38.840 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:38 smithi129 ceph-mon[30979]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-05T01:14:38.840 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:38 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/805232123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T01:14:38.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.849+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7face0108ff0 con 0x7face0106a80 2024-09-05T01:14:38.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.849+0000 7face4890640 1 -- 172.21.15.129:0/2898506912 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7facd809b970 con 0x7face0106a80 2024-09-05T01:14:38.850 INFO:teuthology.orchestra.run.smithi129.stdout:103079215176 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 msgr2=0x7facc80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 0x7facc80a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd0000bc0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 msgr2=0x7face01a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facd800eaa0 tx=0x7facd800ef60 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 shutdown_connections 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facc80a2c90 0x7facc80a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7face0106a80 0x7face01a4260 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7face0101420 0x7face0199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 --2- 172.21.15.129:0/2898506912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7face0100a80 0x7face01991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 >> 172.21.15.129:0/2898506912 conn(0x7face00fc7a0 msgr2=0x7face00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:38.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 shutdown_connections 2024-09-05T01:14:38.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:38.852+0000 7face5892640 1 -- 172.21.15.129:0/2898506912 wait complete. 2024-09-05T01:14:39.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:38 smithi194 ceph-mon[34941]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-05T01:14:39.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:38 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/805232123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T01:14:39.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:38 smithi183 ceph-mon[34589]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-09-05T01:14:39.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:38 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/805232123' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 -- 172.21.15.129:0/931139498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0101de0 msgr2=0x7fe6c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/931139498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0101de0 0x7fe6c01021c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009900 tx=0x7fe6b002a410 comp rx=0 tx=0).stop 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 -- 172.21.15.129:0/931139498 shutdown_connections 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/931139498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0106f10 0x7fe6c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/931139498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0102790 0x7fe6c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/931139498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0101de0 0x7fe6c01021c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 -- 172.21.15.129:0/931139498 >> 172.21.15.129:0/931139498 conn(0x7fe6c00fd8c0 msgr2=0x7fe6c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 -- 172.21.15.129:0/931139498 shutdown_connections 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.287+0000 7fe6c58e9640 1 -- 172.21.15.129:0/931139498 wait complete. 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 Processor -- start 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 -- start start 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0101de0 0x7fe6c00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0106f10 0x7fe6c0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0111e20 con 0x7fe6c0102790 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0111ca0 con 0x7fe6c0101de0 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6c58e9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c0111fa0 con 0x7fe6c0106f10 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6b67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:39.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6b67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55276/0 (socket says 172.21.15.129:55276) 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6b67fc640 1 -- 172.21.15.129:0/1151207464 learned_addr learned my addr 172.21.15.129:0/1151207464 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.288+0000 7fe6bf7fe640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0106f10 0x7fe6c0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6beffd640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0101de0 0x7fe6c00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 -- 172.21.15.129:0/1151207464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0106f10 msgr2=0x7fe6c0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0106f10 0x7fe6c0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 -- 172.21.15.129:0/1151207464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0101de0 msgr2=0x7fe6c00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0101de0 0x7fe6c00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 -- 172.21.15.129:0/1151207464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c01a67d0 con 0x7fe6c0102790 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6b67fc640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800d770 tx=0x7fe6a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8017070 con 0x7fe6c0102790 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a80054d0 con 0x7fe6c0102790 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c01a6ac0 con 0x7fe6c0102790 2024-09-05T01:14:39.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8004cf0 con 0x7fe6c0102790 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.289+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c01a6fb0 con 0x7fe6c0102790 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.290+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c01021c0 con 0x7fe6c0102790 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.291+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe6a8004e90 con 0x7fe6c0102790 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.292+0000 7fe6c48e7640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 0x7fe6ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.292+0000 7fe6beffd640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 0x7fe6ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.292+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe6a80cf4a0 con 0x7fe6c0102790 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.293+0000 7fe6beffd640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 0x7fe6ac0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0031440 tx=0x7fe6b00095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:39.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.295+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6a809bf40 con 0x7fe6c0102790 2024-09-05T01:14:39.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.465+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe6c010a0f0 con 0x7fe6c0102790 2024-09-05T01:14:39.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.465+0000 7fe6c48e7640 1 -- 172.21.15.129:0/1151207464 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6a809b840 con 0x7fe6c0102790 2024-09-05T01:14:39.466 INFO:teuthology.orchestra.run.smithi129.stdout:236223201327 2024-09-05T01:14:39.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.467+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 msgr2=0x7fe6ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:39.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.467+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 0x7fe6ac0a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0031440 tx=0x7fe6b00095f0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.467+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 msgr2=0x7fe6c0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:39.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.467+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800d770 tx=0x7fe6a800dc30 comp rx=0 tx=0).stop 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 shutdown_connections 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6ac0a2c10 0x7fe6ac0a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6c0106f10 0x7fe6c0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6c0102790 0x7fe6c0074a30 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 --2- 172.21.15.129:0/1151207464 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6c0101de0 0x7fe6c00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 >> 172.21.15.129:0/1151207464 conn(0x7fe6c00fd8c0 msgr2=0x7fe6c00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 shutdown_connections 2024-09-05T01:14:39.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:39.468+0000 7fe6c58e9640 1 -- 172.21.15.129:0/1151207464 wait complete. 2024-09-05T01:14:39.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2898506912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T01:14:39.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1151207464' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T01:14:40.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:39 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2898506912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T01:14:40.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:39 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1151207464' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T01:14:40.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:39 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2898506912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T01:14:40.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:39 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1151207464' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T01:14:40.351 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201325 got 236223201327 for osd.8 2024-09-05T01:14:40.351 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 -- 172.21.15.129:0/2748802339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84101de0 msgr2=0x7f1f841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 --2- 172.21.15.129:0/2748802339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84101de0 0x7f1f841021c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009900 tx=0x7f1f7402a410 comp rx=0 tx=0).stop 2024-09-05T01:14:40.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 -- 172.21.15.129:0/2748802339 shutdown_connections 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 --2- 172.21.15.129:0/2748802339 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84106f10 0x7f1f8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 --2- 172.21.15.129:0/2748802339 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84102790 0x7f1f841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 --2- 172.21.15.129:0/2748802339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84101de0 0x7f1f841021c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 -- 172.21.15.129:0/2748802339 >> 172.21.15.129:0/2748802339 conn(0x7f1f840fd8c0 msgr2=0x7f1f840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 -- 172.21.15.129:0/2748802339 shutdown_connections 2024-09-05T01:14:40.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.529+0000 7f1f89893640 1 -- 172.21.15.129:0/2748802339 wait complete. 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 Processor -- start 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 -- start start 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 0x7f1f84198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84106f10 0x7f1f8419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84111e20 con 0x7f1f84102790 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84111ca0 con 0x7f1f84106f10 2024-09-05T01:14:40.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f89893640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f84111fa0 con 0x7f1f84101de0 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55302/0 (socket says 172.21.15.129:55302) 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f827fc640 1 -- 172.21.15.129:0/3075371477 learned_addr learned my addr 172.21.15.129:0/3075371477 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f82ffd640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 0x7f1f84198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.530+0000 7f1f837fe640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84106f10 0x7f1f8419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 -- 172.21.15.129:0/3075371477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 msgr2=0x7f1f84198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 0x7f1f84198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 -- 172.21.15.129:0/3075371477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84106f10 msgr2=0x7f1f8419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84106f10 0x7f1f8419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 -- 172.21.15.129:0/3075371477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8419d330 con 0x7f1f84102790 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f82ffd640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 0x7f1f84198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f827fc640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800d950 tx=0x7f1f7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:40.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f78017070 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8419d620 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f78009c30 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7800f620 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.531+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841a4f00 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.532+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f841021c0 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.533+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1f780109a0 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.534+0000 7f1f88891640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 0x7f1f6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.534+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1f780cf1f0 con 0x7f1f84102790 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.534+0000 7f1f82ffd640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 0x7f1f6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.534+0000 7f1f82ffd640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 0x7f1f6c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1f74031440 tx=0x7f1f74005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:40.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.537+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f78009de0 con 0x7f1f84102790 2024-09-05T01:14:40.707 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.707+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1f841996e0 con 0x7f1f84102790 2024-09-05T01:14:40.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.707+0000 7f1f88891640 1 -- 172.21.15.129:0/3075371477 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1f841996e0 con 0x7f1f84102790 2024-09-05T01:14:40.708 INFO:teuthology.orchestra.run.smithi129.stdout:133143986244 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 msgr2=0x7f1f6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 0x7f1f6c0a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1f74031440 tx=0x7f1f74005a20 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 msgr2=0x7f1f841988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800d950 tx=0x7f1f7800de10 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 shutdown_connections 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f6c0a2c10 0x7f1f6c0a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f84106f10 0x7f1f8419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102790 0x7f1f841988c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 --2- 172.21.15.129:0/3075371477 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f84101de0 0x7f1f84198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 >> 172.21.15.129:0/3075371477 conn(0x7f1f840fd8c0 msgr2=0x7f1f840fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:40.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 shutdown_connections 2024-09-05T01:14:40.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:40.709+0000 7f1f89893640 1 -- 172.21.15.129:0/3075371477 wait complete. 2024-09-05T01:14:40.861 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:40 smithi129 ceph-mon[30979]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-05T01:14:40.888 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215173 got 103079215176 for osd.3 2024-09-05T01:14:40.888 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:41.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:40 smithi194 ceph-mon[34941]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-05T01:14:41.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:40 smithi183 ceph-mon[34589]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-09-05T01:14:41.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411402 got 77309411405 for osd.2 2024-09-05T01:14:41.334 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:41.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:41 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3075371477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T01:14:42.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:41 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3075371477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T01:14:42.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:41 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3075371477' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T01:14:42.169 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986239 got 133143986244 for osd.4 2024-09-05T01:14:42.169 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:42.774 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738454 for osd.0 2024-09-05T01:14:42.774 DEBUG:teuthology.parallel:result is None 2024-09-05T01:14:42.774 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T01:14:42.775 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:14:42.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:42 smithi129 ceph-mon[30979]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:43.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:42 smithi194 ceph-mon[34941]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:43.029 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:43.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:42 smithi183 ceph-mon[34589]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 -- 172.21.15.129:0/1115420468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1082a0 msgr2=0x7f1e1c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 --2- 172.21.15.129:0/1115420468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c10ec40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e14009cd0 tx=0x7f1e1402a910 comp rx=0 tx=0).stop 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 -- 172.21.15.129:0/1115420468 shutdown_connections 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 --2- 172.21.15.129:0/1115420468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c10ec40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 --2- 172.21.15.129:0/1115420468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 --2- 172.21.15.129:0/1115420468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c102d30 0x7f1e1c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 -- 172.21.15.129:0/1115420468 >> 172.21.15.129:0/1115420468 conn(0x7f1e1c0fc7d0 msgr2=0x7f1e1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 -- 172.21.15.129:0/1115420468 shutdown_connections 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.483+0000 7f1e21abf640 1 -- 172.21.15.129:0/1115420468 wait complete. 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 Processor -- start 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 -- start start 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 0x7f1e1c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c112010 con 0x7f1e1c1036e0 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c111e90 con 0x7f1e1c1082a0 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e21abf640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1e1c112190 con 0x7f1e1c102d30 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:43.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47050/0 (socket says 172.21.15.129:47050) 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e13fff640 1 -- 172.21.15.129:0/3376795249 learned_addr learned my addr 172.21.15.129:0/3376795249 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e1b7fe640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 0x7f1e1c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e1bfff640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.484+0000 7f1e13fff640 1 -- 172.21.15.129:0/3376795249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 msgr2=0x7f1e1c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e13fff640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 0x7f1e1c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e13fff640 1 -- 172.21.15.129:0/3376795249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 msgr2=0x7f1e1c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e13fff640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e13fff640 1 -- 172.21.15.129:0/3376795249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e1c19d580 con 0x7f1e1c1036e0 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e1b7fe640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 0x7f1e1c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e1bfff640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e13fff640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e04006fd0 tx=0x7f1e0400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04011830 con 0x7f1e1c1036e0 2024-09-05T01:14:43.498 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1e04016070 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e1c19d870 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04010df0 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.485+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c1a5150 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.486+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e1c103190 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.487+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1e040119d0 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.488+0000 7f1e197fa640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 0x7f1e080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.488+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1e040cf210 con 0x7f1e1c1036e0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.488+0000 7f1e1b7fe640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 0x7f1e080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.488+0000 7f1e1b7fe640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 0x7f1e080a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:43.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.491+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e0409bdc0 con 0x7f1e1c1036e0 2024-09-05T01:14:43.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.656+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e1c10a2e0 con 0x7f1e080a2c10 2024-09-05T01:14:43.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.664+0000 7f1e197fa640 1 -- 172.21.15.129:0/3376795249 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340344 (secure 0 0 0) 0x7f1e1c10a2e0 con 0x7f1e080a2c10 2024-09-05T01:14:43.669 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:43.670 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.668+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 msgr2=0x7f1e080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 0x7f1e080a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c006fd0 tx=0x7f1e0c008040 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 msgr2=0x7f1e1c198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1e04006fd0 tx=0x7f1e0400b8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 shutdown_connections 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1e080a2c10 0x7f1e080a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1e1c1082a0 0x7f1e1c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1e1c1036e0 0x7f1e1c198b10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 --2- 172.21.15.129:0/3376795249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1e1c102d30 0x7f1e1c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 >> 172.21.15.129:0/3376795249 conn(0x7f1e1c0fc7d0 msgr2=0x7f1e1c105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 shutdown_connections 2024-09-05T01:14:43.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:43.669+0000 7f1e21abf640 1 -- 172.21.15.129:0/3376795249 wait complete. 2024-09-05T01:14:44.352 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-05T01:14:41.771226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":448,"num_read_kb":341,"num_write":350,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392248,"kb_used_data":13796,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549576,"statfs":{"total":1151940427776,"available":1151538765824,"internally_reserved":0,"allocated":14127104,"data_stored":7892513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003545"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831970+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:14:18.831970+0000","last_peered":"2024-09-05T01:14:18.831970+0000","last_clean":"2024-09-05T01:14:18.831970+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:14:18.831970+0000","last_undegraded":"2024-09-05T01:14:18.831970+0000","last_fullsized":"2024-09-05T01:14:18.831970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911065+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:14:18.911065+0000","last_peered":"2024-09-05T01:14:18.911065+0000","last_clean":"2024-09-05T01:14:18.911065+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:14:18.911065+0000","last_undegraded":"2024-09-05T01:14:18.911065+0000","last_fullsized":"2024-09-05T01:14:18.911065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910723+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:14:18.910723+0000","last_peered":"2024-09-05T01:14:18.910723+0000","last_clean":"2024-09-05T01:14:18.910723+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:14:18.910723+0000","last_undegraded":"2024-09-05T01:14:18.910723+0000","last_fullsized":"2024-09-05T01:14:18.910723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911282+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:14:18.911282+0000","last_peered":"2024-09-05T01:14:18.911282+0000","last_clean":"2024-09-05T01:14:18.911282+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:14:18.911282+0000","last_undegraded":"2024-09-05T01:14:18.911282+0000","last_fullsized":"2024-09-05T01:14:18.911282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834737+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:14:18.834737+0000","last_peered":"2024-09-05T01:14:18.834737+0000","last_clean":"2024-09-05T01:14:18.834737+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:14:18.834737+0000","last_undegraded":"2024-09-05T01:14:18.834737+0000","last_fullsized":"2024-09-05T01:14:18.834737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913937+0000","last_change":"2024-09-05T01:14:18.913937+0000","last_active":"2024-09-05T01:14:18.913937+0000","last_peered":"2024-09-05T01:14:18.913937+0000","last_clean":"2024-09-05T01:14:18.913937+0000","last_became_active":"2024-09-05T01:14:18.913425+0000","last_became_peered":"2024-09-05T01:14:18.913425+0000","last_unstale":"2024-09-05T01:14:18.913937+0000","last_undegraded":"2024-09-05T01:14:18.913937+0000","last_fullsized":"2024-09-05T01:14:18.913937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:56.174787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839094+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:14:18.839094+0000","last_peered":"2024-09-05T01:14:18.839094+0000","last_clean":"2024-09-05T01:14:18.839094+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:14:18.839094+0000","last_undegraded":"2024-09-05T01:14:18.839094+0000","last_fullsized":"2024-09-05T01:14:18.839094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911672+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:14:18.911672+0000","last_peered":"2024-09-05T01:14:18.911672+0000","last_clean":"2024-09-05T01:14:18.911672+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:14:18.911672+0000","last_undegraded":"2024-09-05T01:14:18.911672+0000","last_fullsized":"2024-09-05T01:14:18.911672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833934+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:14:18.833934+0000","last_peered":"2024-09-05T01:14:18.833934+0000","last_clean":"2024-09-05T01:14:18.833934+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:14:18.833934+0000","last_undegraded":"2024-09-05T01:14:18.833934+0000","last_fullsized":"2024-09-05T01:14:18.833934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839216+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:14:18.839216+0000","last_peered":"2024-09-05T01:14:18.839216+0000","last_clean":"2024-09-05T01:14:18.839216+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:14:18.839216+0000","last_undegraded":"2024-09-05T01:14:18.839216+0000","last_fullsized":"2024-09-05T01:14:18.839216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913351+0000","last_change":"2024-09-05T01:14:18.913351+0000","last_active":"2024-09-05T01:14:18.913351+0000","last_peered":"2024-09-05T01:14:18.913351+0000","last_clean":"2024-09-05T01:14:18.913351+0000","last_became_active":"2024-09-05T01:14:18.912935+0000","last_became_peered":"2024-09-05T01:14:18.912935+0000","last_unstale":"2024-09-05T01:14:18.913351+0000","last_undegraded":"2024-09-05T01:14:18.913351+0000","last_fullsized":"2024-09-05T01:14:18.913351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:32:30.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833180+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:14:18.833180+0000","last_peered":"2024-09-05T01:14:18.833180+0000","last_clean":"2024-09-05T01:14:18.833180+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:14:18.833180+0000","last_undegraded":"2024-09-05T01:14:18.833180+0000","last_fullsized":"2024-09-05T01:14:18.833180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912209+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:14:18.912209+0000","last_peered":"2024-09-05T01:14:18.912209+0000","last_clean":"2024-09-05T01:14:18.912209+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:14:18.912209+0000","last_undegraded":"2024-09-05T01:14:18.912209+0000","last_fullsized":"2024-09-05T01:14:18.912209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838781+0000","last_change":"2024-09-05T01:13:03.882357+0000","last_active":"2024-09-05T01:14:18.838781+0000","last_peered":"2024-09-05T01:14:18.838781+0000","last_clean":"2024-09-05T01:14:18.838781+0000","last_became_active":"2024-09-05T01:13:03.881983+0000","last_became_peered":"2024-09-05T01:13:03.881983+0000","last_unstale":"2024-09-05T01:14:18.838781+0000","last_undegraded":"2024-09-05T01:14:18.838781+0000","last_fullsized":"2024-09-05T01:14:18.838781+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:09:44.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833493+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:14:18.833493+0000","last_peered":"2024-09-05T01:14:18.833493+0000","last_clean":"2024-09-05T01:14:18.833493+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:14:18.833493+0000","last_undegraded":"2024-09-05T01:14:18.833493+0000","last_fullsized":"2024-09-05T01:14:18.833493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911268+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:14:18.911268+0000","last_peered":"2024-09-05T01:14:18.911268+0000","last_clean":"2024-09-05T01:14:18.911268+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:14:18.911268+0000","last_undegraded":"2024-09-05T01:14:18.911268+0000","last_fullsized":"2024-09-05T01:14:18.911268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910837+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:14:18.910837+0000","last_peered":"2024-09-05T01:14:18.910837+0000","last_clean":"2024-09-05T01:14:18.910837+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:14:18.910837+0000","last_undegraded":"2024-09-05T01:14:18.910837+0000","last_fullsized":"2024-09-05T01:14:18.910837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833763+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:14:18.833763+0000","last_peered":"2024-09-05T01:14:18.833763+0000","last_clean":"2024-09-05T01:14:18.833763+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:14:18.833763+0000","last_undegraded":"2024-09-05T01:14:18.833763+0000","last_fullsized":"2024-09-05T01:14:18.833763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834435+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:14:18.834435+0000","last_peered":"2024-09-05T01:14:18.834435+0000","last_clean":"2024-09-05T01:14:18.834435+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:14:18.834435+0000","last_undegraded":"2024-09-05T01:14:18.834435+0000","last_fullsized":"2024-09-05T01:14:18.834435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911382+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:14:18.911382+0000","last_peered":"2024-09-05T01:14:18.911382+0000","last_clean":"2024-09-05T01:14:18.911382+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:14:18.911382+0000","last_undegraded":"2024-09-05T01:14:18.911382+0000","last_fullsized":"2024-09-05T01:14:18.911382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831810+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:14:18.831810+0000","last_peered":"2024-09-05T01:14:18.831810+0000","last_clean":"2024-09-05T01:14:18.831810+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:14:18.831810+0000","last_undegraded":"2024-09-05T01:14:18.831810+0000","last_fullsized":"2024-09-05T01:14:18.831810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838595+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:14:18.838595+0000","last_peered":"2024-09-05T01:14:18.838595+0000","last_clean":"2024-09-05T01:14:18.838595+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:14:18.838595+0000","last_undegraded":"2024-09-05T01:14:18.838595+0000","last_fullsized":"2024-09-05T01:14:18.838595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911346+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:14:18.911346+0000","last_peered":"2024-09-05T01:14:18.911346+0000","last_clean":"2024-09-05T01:14:18.911346+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:14:18.911346+0000","last_undegraded":"2024-09-05T01:14:18.911346+0000","last_fullsized":"2024-09-05T01:14:18.911346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833601+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:14:18.833601+0000","last_peered":"2024-09-05T01:14:18.833601+0000","last_clean":"2024-09-05T01:14:18.833601+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:14:18.833601+0000","last_undegraded":"2024-09-05T01:14:18.833601+0000","last_fullsized":"2024-09-05T01:14:18.833601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912689+0000","last_change":"2024-09-05T01:14:18.912689+0000","last_active":"2024-09-05T01:14:18.912689+0000","last_peered":"2024-09-05T01:14:18.912689+0000","last_clean":"2024-09-05T01:14:18.912689+0000","last_became_active":"2024-09-05T01:14:18.912358+0000","last_became_peered":"2024-09-05T01:14:18.912358+0000","last_unstale":"2024-09-05T01:14:18.912689+0000","last_undegraded":"2024-09-05T01:14:18.912689+0000","last_fullsized":"2024-09-05T01:14:18.912689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:59.876348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834488+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:14:18.834488+0000","last_peered":"2024-09-05T01:14:18.834488+0000","last_clean":"2024-09-05T01:14:18.834488+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:14:18.834488+0000","last_undegraded":"2024-09-05T01:14:18.834488+0000","last_fullsized":"2024-09-05T01:14:18.834488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910736+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:14:18.910736+0000","last_peered":"2024-09-05T01:14:18.910736+0000","last_clean":"2024-09-05T01:14:18.910736+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:14:18.910736+0000","last_undegraded":"2024-09-05T01:14:18.910736+0000","last_fullsized":"2024-09-05T01:14:18.910736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911051+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:14:18.911051+0000","last_peered":"2024-09-05T01:14:18.911051+0000","last_clean":"2024-09-05T01:14:18.911051+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:14:18.911051+0000","last_undegraded":"2024-09-05T01:14:18.911051+0000","last_fullsized":"2024-09-05T01:14:18.911051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831875+0000","last_change":"2024-09-05T01:13:03.873898+0000","last_active":"2024-09-05T01:14:18.831875+0000","last_peered":"2024-09-05T01:14:18.831875+0000","last_clean":"2024-09-05T01:14:18.831875+0000","last_became_active":"2024-09-05T01:13:03.873546+0000","last_became_peered":"2024-09-05T01:13:03.873546+0000","last_unstale":"2024-09-05T01:14:18.831875+0000","last_undegraded":"2024-09-05T01:14:18.831875+0000","last_fullsized":"2024-09-05T01:14:18.831875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:45:48.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910909+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:14:18.910909+0000","last_peered":"2024-09-05T01:14:18.910909+0000","last_clean":"2024-09-05T01:14:18.910909+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:14:18.910909+0000","last_undegraded":"2024-09-05T01:14:18.910909+0000","last_fullsized":"2024-09-05T01:14:18.910909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834477+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:14:18.834477+0000","last_peered":"2024-09-05T01:14:18.834477+0000","last_clean":"2024-09-05T01:14:18.834477+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:14:18.834477+0000","last_undegraded":"2024-09-05T01:14:18.834477+0000","last_fullsized":"2024-09-05T01:14:18.834477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839192+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:14:18.839192+0000","last_peered":"2024-09-05T01:14:18.839192+0000","last_clean":"2024-09-05T01:14:18.839192+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:14:18.839192+0000","last_undegraded":"2024-09-05T01:14:18.839192+0000","last_fullsized":"2024-09-05T01:14:18.839192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834417+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:14:18.834417+0000","last_peered":"2024-09-05T01:14:18.834417+0000","last_clean":"2024-09-05T01:14:18.834417+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:14:18.834417+0000","last_undegraded":"2024-09-05T01:14:18.834417+0000","last_fullsized":"2024-09-05T01:14:18.834417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833148+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:14:18.833148+0000","last_peered":"2024-09-05T01:14:18.833148+0000","last_clean":"2024-09-05T01:14:18.833148+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:14:18.833148+0000","last_undegraded":"2024-09-05T01:14:18.833148+0000","last_fullsized":"2024-09-05T01:14:18.833148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831415+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:14:18.831415+0000","last_peered":"2024-09-05T01:14:18.831415+0000","last_clean":"2024-09-05T01:14:18.831415+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:14:18.831415+0000","last_undegraded":"2024-09-05T01:14:18.831415+0000","last_fullsized":"2024-09-05T01:14:18.831415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838813+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:14:18.838813+0000","last_peered":"2024-09-05T01:14:18.838813+0000","last_clean":"2024-09-05T01:14:18.838813+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:14:18.838813+0000","last_undegraded":"2024-09-05T01:14:18.838813+0000","last_fullsized":"2024-09-05T01:14:18.838813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.832774+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:14:18.832774+0000","last_peered":"2024-09-05T01:14:18.832774+0000","last_clean":"2024-09-05T01:14:18.832774+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:14:18.832774+0000","last_undegraded":"2024-09-05T01:14:18.832774+0000","last_fullsized":"2024-09-05T01:14:18.832774+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912415+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:14:18.912415+0000","last_peered":"2024-09-05T01:14:18.912415+0000","last_clean":"2024-09-05T01:14:18.912415+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:14:18.912415+0000","last_undegraded":"2024-09-05T01:14:18.912415+0000","last_fullsized":"2024-09-05T01:14:18.912415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834243+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:14:18.834243+0000","last_peered":"2024-09-05T01:14:18.834243+0000","last_clean":"2024-09-05T01:14:18.834243+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:14:18.834243+0000","last_undegraded":"2024-09-05T01:14:18.834243+0000","last_fullsized":"2024-09-05T01:14:18.834243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912776+0000","last_change":"2024-09-05T01:14:18.912776+0000","last_active":"2024-09-05T01:14:18.912776+0000","last_peered":"2024-09-05T01:14:18.912776+0000","last_clean":"2024-09-05T01:14:18.912776+0000","last_became_active":"2024-09-05T01:14:18.912337+0000","last_became_peered":"2024-09-05T01:14:18.912337+0000","last_unstale":"2024-09-05T01:14:18.912776+0000","last_undegraded":"2024-09-05T01:14:18.912776+0000","last_fullsized":"2024-09-05T01:14:18.912776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:37:57.935580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:36.925837+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:14:36.925837+0000","last_peered":"2024-09-05T01:14:36.925837+0000","last_clean":"2024-09-05T01:14:36.925837+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:14:36.925837+0000","last_undegraded":"2024-09-05T01:14:36.925837+0000","last_fullsized":"2024-09-05T01:14:36.925837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834288+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:14:18.834288+0000","last_peered":"2024-09-05T01:14:18.834288+0000","last_clean":"2024-09-05T01:14:18.834288+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:14:18.834288+0000","last_undegraded":"2024-09-05T01:14:18.834288+0000","last_fullsized":"2024-09-05T01:14:18.834288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833762+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:14:18.833762+0000","last_peered":"2024-09-05T01:14:18.833762+0000","last_clean":"2024-09-05T01:14:18.833762+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:14:18.833762+0000","last_undegraded":"2024-09-05T01:14:18.833762+0000","last_fullsized":"2024-09-05T01:14:18.833762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911783+0000","last_change":"2024-09-05T01:13:03.880884+0000","last_active":"2024-09-05T01:14:18.911783+0000","last_peered":"2024-09-05T01:14:18.911783+0000","last_clean":"2024-09-05T01:14:18.911783+0000","last_became_active":"2024-09-05T01:13:03.880274+0000","last_became_peered":"2024-09-05T01:13:03.880274+0000","last_unstale":"2024-09-05T01:14:18.911783+0000","last_undegraded":"2024-09-05T01:14:18.911783+0000","last_fullsized":"2024-09-05T01:14:18.911783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:24.973934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836621+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:14:18.836621+0000","last_peered":"2024-09-05T01:14:18.836621+0000","last_clean":"2024-09-05T01:14:18.836621+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:14:18.836621+0000","last_undegraded":"2024-09-05T01:14:18.836621+0000","last_fullsized":"2024-09-05T01:14:18.836621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910805+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:14:18.910805+0000","last_peered":"2024-09-05T01:14:18.910805+0000","last_clean":"2024-09-05T01:14:18.910805+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:14:18.910805+0000","last_undegraded":"2024-09-05T01:14:18.910805+0000","last_fullsized":"2024-09-05T01:14:18.910805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834236+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:14:18.834236+0000","last_peered":"2024-09-05T01:14:18.834236+0000","last_clean":"2024-09-05T01:14:18.834236+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:14:18.834236+0000","last_undegraded":"2024-09-05T01:14:18.834236+0000","last_fullsized":"2024-09-05T01:14:18.834236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911981+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:14:18.911981+0000","last_peered":"2024-09-05T01:14:18.911981+0000","last_clean":"2024-09-05T01:14:18.911981+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:14:18.911981+0000","last_undegraded":"2024-09-05T01:14:18.911981+0000","last_fullsized":"2024-09-05T01:14:18.911981+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912424+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:14:18.912424+0000","last_peered":"2024-09-05T01:14:18.912424+0000","last_clean":"2024-09-05T01:14:18.912424+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:14:18.912424+0000","last_undegraded":"2024-09-05T01:14:18.912424+0000","last_fullsized":"2024-09-05T01:14:18.912424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912677+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:14:18.912677+0000","last_peered":"2024-09-05T01:14:18.912677+0000","last_clean":"2024-09-05T01:14:18.912677+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:14:18.912677+0000","last_undegraded":"2024-09-05T01:14:18.912677+0000","last_fullsized":"2024-09-05T01:14:18.912677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911573+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:14:18.911573+0000","last_peered":"2024-09-05T01:14:18.911573+0000","last_clean":"2024-09-05T01:14:18.911573+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:14:18.911573+0000","last_undegraded":"2024-09-05T01:14:18.911573+0000","last_fullsized":"2024-09-05T01:14:18.911573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910751+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:14:18.910751+0000","last_peered":"2024-09-05T01:14:18.910751+0000","last_clean":"2024-09-05T01:14:18.910751+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:14:18.910751+0000","last_undegraded":"2024-09-05T01:14:18.910751+0000","last_fullsized":"2024-09-05T01:14:18.910751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912624+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:14:18.912624+0000","last_peered":"2024-09-05T01:14:18.912624+0000","last_clean":"2024-09-05T01:14:18.912624+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:14:18.912624+0000","last_undegraded":"2024-09-05T01:14:18.912624+0000","last_fullsized":"2024-09-05T01:14:18.912624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837100+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:14:18.837100+0000","last_peered":"2024-09-05T01:14:18.837100+0000","last_clean":"2024-09-05T01:14:18.837100+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:14:18.837100+0000","last_undegraded":"2024-09-05T01:14:18.837100+0000","last_fullsized":"2024-09-05T01:14:18.837100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834531+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:14:18.834531+0000","last_peered":"2024-09-05T01:14:18.834531+0000","last_clean":"2024-09-05T01:14:18.834531+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:14:18.834531+0000","last_undegraded":"2024-09-05T01:14:18.834531+0000","last_fullsized":"2024-09-05T01:14:18.834531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912761+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:14:18.912761+0000","last_peered":"2024-09-05T01:14:18.912761+0000","last_clean":"2024-09-05T01:14:18.912761+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:14:18.912761+0000","last_undegraded":"2024-09-05T01:14:18.912761+0000","last_fullsized":"2024-09-05T01:14:18.912761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834786+0000","last_change":"2024-09-05T01:13:03.877073+0000","last_active":"2024-09-05T01:14:18.834786+0000","last_peered":"2024-09-05T01:14:18.834786+0000","last_clean":"2024-09-05T01:14:18.834786+0000","last_became_active":"2024-09-05T01:13:03.876801+0000","last_became_peered":"2024-09-05T01:13:03.876801+0000","last_unstale":"2024-09-05T01:14:18.834786+0000","last_undegraded":"2024-09-05T01:14:18.834786+0000","last_fullsized":"2024-09-05T01:14:18.834786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:13:24.959411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.914428+0000","last_change":"2024-09-05T01:14:18.913738+0000","last_active":"2024-09-05T01:14:18.914428+0000","last_peered":"2024-09-05T01:14:18.914428+0000","last_clean":"2024-09-05T01:14:18.914428+0000","last_became_active":"2024-09-05T01:14:18.913084+0000","last_became_peered":"2024-09-05T01:14:18.913084+0000","last_unstale":"2024-09-05T01:14:18.914428+0000","last_undegraded":"2024-09-05T01:14:18.914428+0000","last_fullsized":"2024-09-05T01:14:18.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:43.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835189+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:14:18.835189+0000","last_peered":"2024-09-05T01:14:18.835189+0000","last_clean":"2024-09-05T01:14:18.835189+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:14:18.835189+0000","last_undegraded":"2024-09-05T01:14:18.835189+0000","last_fullsized":"2024-09-05T01:14:18.835189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838959+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:14:18.838959+0000","last_peered":"2024-09-05T01:14:18.838959+0000","last_clean":"2024-09-05T01:14:18.838959+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:14:18.838959+0000","last_undegraded":"2024-09-05T01:14:18.838959+0000","last_fullsized":"2024-09-05T01:14:18.838959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836008+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:14:18.836008+0000","last_peered":"2024-09-05T01:14:18.836008+0000","last_clean":"2024-09-05T01:14:18.836008+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:14:18.836008+0000","last_undegraded":"2024-09-05T01:14:18.836008+0000","last_fullsized":"2024-09-05T01:14:18.836008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836188+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:14:18.836188+0000","last_peered":"2024-09-05T01:14:18.836188+0000","last_clean":"2024-09-05T01:14:18.836188+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:14:18.836188+0000","last_undegraded":"2024-09-05T01:14:18.836188+0000","last_fullsized":"2024-09-05T01:14:18.836188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840016+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:14:18.840016+0000","last_peered":"2024-09-05T01:14:18.840016+0000","last_clean":"2024-09-05T01:14:18.840016+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:14:18.840016+0000","last_undegraded":"2024-09-05T01:14:18.840016+0000","last_fullsized":"2024-09-05T01:14:18.840016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834690+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:14:18.834690+0000","last_peered":"2024-09-05T01:14:18.834690+0000","last_clean":"2024-09-05T01:14:18.834690+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:14:18.834690+0000","last_undegraded":"2024-09-05T01:14:18.834690+0000","last_fullsized":"2024-09-05T01:14:18.834690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911622+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:14:18.911622+0000","last_peered":"2024-09-05T01:14:18.911622+0000","last_clean":"2024-09-05T01:14:18.911622+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:14:18.911622+0000","last_undegraded":"2024-09-05T01:14:18.911622+0000","last_fullsized":"2024-09-05T01:14:18.911622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":87,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:34.923481+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:14:34.923481+0000","last_peered":"2024-09-05T01:14:34.923481+0000","last_clean":"2024-09-05T01:14:34.923481+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:14:34.923481+0000","last_undegraded":"2024-09-05T01:14:34.923481+0000","last_fullsized":"2024-09-05T01:14:34.923481+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831873+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:14:18.831873+0000","last_peered":"2024-09-05T01:14:18.831873+0000","last_clean":"2024-09-05T01:14:18.831873+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:14:18.831873+0000","last_undegraded":"2024-09-05T01:14:18.831873+0000","last_fullsized":"2024-09-05T01:14:18.831873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912320+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:14:18.912320+0000","last_peered":"2024-09-05T01:14:18.912320+0000","last_clean":"2024-09-05T01:14:18.912320+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:14:18.912320+0000","last_undegraded":"2024-09-05T01:14:18.912320+0000","last_fullsized":"2024-09-05T01:14:18.912320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837195+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:14:18.837195+0000","last_peered":"2024-09-05T01:14:18.837195+0000","last_clean":"2024-09-05T01:14:18.837195+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:14:18.837195+0000","last_undegraded":"2024-09-05T01:14:18.837195+0000","last_fullsized":"2024-09-05T01:14:18.837195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834794+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:14:18.834794+0000","last_peered":"2024-09-05T01:14:18.834794+0000","last_clean":"2024-09-05T01:14:18.834794+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:14:18.834794+0000","last_undegraded":"2024-09-05T01:14:18.834794+0000","last_fullsized":"2024-09-05T01:14:18.834794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834032+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:14:18.834032+0000","last_peered":"2024-09-05T01:14:18.834032+0000","last_clean":"2024-09-05T01:14:18.834032+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:14:18.834032+0000","last_undegraded":"2024-09-05T01:14:18.834032+0000","last_fullsized":"2024-09-05T01:14:18.834032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911479+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:14:18.911479+0000","last_peered":"2024-09-05T01:14:18.911479+0000","last_clean":"2024-09-05T01:14:18.911479+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:14:18.911479+0000","last_undegraded":"2024-09-05T01:14:18.911479+0000","last_fullsized":"2024-09-05T01:14:18.911479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836716+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:14:18.836716+0000","last_peered":"2024-09-05T01:14:18.836716+0000","last_clean":"2024-09-05T01:14:18.836716+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:14:18.836716+0000","last_undegraded":"2024-09-05T01:14:18.836716+0000","last_fullsized":"2024-09-05T01:14:18.836716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838961+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:14:18.838961+0000","last_peered":"2024-09-05T01:14:18.838961+0000","last_clean":"2024-09-05T01:14:18.838961+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:14:18.838961+0000","last_undegraded":"2024-09-05T01:14:18.838961+0000","last_fullsized":"2024-09-05T01:14:18.838961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":173,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:36.926358+0000","last_change":"2024-09-05T01:13:03.883605+0000","last_active":"2024-09-05T01:14:36.926358+0000","last_peered":"2024-09-05T01:14:36.926358+0000","last_clean":"2024-09-05T01:14:36.926358+0000","last_became_active":"2024-09-05T01:13:03.883275+0000","last_became_peered":"2024-09-05T01:13:03.883275+0000","last_unstale":"2024-09-05T01:14:36.926358+0000","last_undegraded":"2024-09-05T01:14:36.926358+0000","last_fullsized":"2024-09-05T01:14:36.926358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:58:39.763930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":107,"num_read_kb":114,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840367+0000","last_change":"2024-09-05T01:14:18.840367+0000","last_active":"2024-09-05T01:14:18.840367+0000","last_peered":"2024-09-05T01:14:18.840367+0000","last_clean":"2024-09-05T01:14:18.840367+0000","last_became_active":"2024-09-05T01:14:18.839937+0000","last_became_peered":"2024-09-05T01:14:18.839937+0000","last_unstale":"2024-09-05T01:14:18.840367+0000","last_undegraded":"2024-09-05T01:14:18.840367+0000","last_fullsized":"2024-09-05T01:14:18.840367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:49:14.286631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911766+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:14:18.911766+0000","last_peered":"2024-09-05T01:14:18.911766+0000","last_clean":"2024-09-05T01:14:18.911766+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:14:18.911766+0000","last_undegraded":"2024-09-05T01:14:18.911766+0000","last_fullsized":"2024-09-05T01:14:18.911766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836975+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:14:18.836975+0000","last_peered":"2024-09-05T01:14:18.836975+0000","last_clean":"2024-09-05T01:14:18.836975+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:14:18.836975+0000","last_undegraded":"2024-09-05T01:14:18.836975+0000","last_fullsized":"2024-09-05T01:14:18.836975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913011+0000","last_change":"2024-09-05T01:14:18.913011+0000","last_active":"2024-09-05T01:14:18.913011+0000","last_peered":"2024-09-05T01:14:18.913011+0000","last_clean":"2024-09-05T01:14:18.913011+0000","last_became_active":"2024-09-05T01:14:18.912546+0000","last_became_peered":"2024-09-05T01:14:18.912546+0000","last_unstale":"2024-09-05T01:14:18.913011+0000","last_undegraded":"2024-09-05T01:14:18.913011+0000","last_fullsized":"2024-09-05T01:14:18.913011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:11:02.202754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834826+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:14:18.834826+0000","last_peered":"2024-09-05T01:14:18.834826+0000","last_clean":"2024-09-05T01:14:18.834826+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:14:18.834826+0000","last_undegraded":"2024-09-05T01:14:18.834826+0000","last_fullsized":"2024-09-05T01:14:18.834826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912511+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:14:18.912511+0000","last_peered":"2024-09-05T01:14:18.912511+0000","last_clean":"2024-09-05T01:14:18.912511+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:14:18.912511+0000","last_undegraded":"2024-09-05T01:14:18.912511+0000","last_fullsized":"2024-09-05T01:14:18.912511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.830938+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:14:18.830938+0000","last_peered":"2024-09-05T01:14:18.830938+0000","last_clean":"2024-09-05T01:14:18.830938+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:14:18.830938+0000","last_undegraded":"2024-09-05T01:14:18.830938+0000","last_fullsized":"2024-09-05T01:14:18.830938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911089+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:14:18.911089+0000","last_peered":"2024-09-05T01:14:18.911089+0000","last_clean":"2024-09-05T01:14:18.911089+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:14:18.911089+0000","last_undegraded":"2024-09-05T01:14:18.911089+0000","last_fullsized":"2024-09-05T01:14:18.911089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":86,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:35.924852+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:14:35.924852+0000","last_peered":"2024-09-05T01:14:35.924852+0000","last_clean":"2024-09-05T01:14:35.924852+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:14:35.924852+0000","last_undegraded":"2024-09-05T01:14:35.924852+0000","last_fullsized":"2024-09-05T01:14:35.924852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910667+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:14:18.910667+0000","last_peered":"2024-09-05T01:14:18.910667+0000","last_clean":"2024-09-05T01:14:18.910667+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:14:18.910667+0000","last_undegraded":"2024-09-05T01:14:18.910667+0000","last_fullsized":"2024-09-05T01:14:18.910667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834661+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:14:18.834661+0000","last_peered":"2024-09-05T01:14:18.834661+0000","last_clean":"2024-09-05T01:14:18.834661+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:14:18.834661+0000","last_undegraded":"2024-09-05T01:14:18.834661+0000","last_fullsized":"2024-09-05T01:14:18.834661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831261+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:14:18.831261+0000","last_peered":"2024-09-05T01:14:18.831261+0000","last_clean":"2024-09-05T01:14:18.831261+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:14:18.831261+0000","last_undegraded":"2024-09-05T01:14:18.831261+0000","last_fullsized":"2024-09-05T01:14:18.831261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837140+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:14:18.837140+0000","last_peered":"2024-09-05T01:14:18.837140+0000","last_clean":"2024-09-05T01:14:18.837140+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:14:18.837140+0000","last_undegraded":"2024-09-05T01:14:18.837140+0000","last_fullsized":"2024-09-05T01:14:18.837140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835239+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:14:18.835239+0000","last_peered":"2024-09-05T01:14:18.835239+0000","last_clean":"2024-09-05T01:14:18.835239+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:14:18.835239+0000","last_undegraded":"2024-09-05T01:14:18.835239+0000","last_fullsized":"2024-09-05T01:14:18.835239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911108+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:14:18.911108+0000","last_peered":"2024-09-05T01:14:18.911108+0000","last_clean":"2024-09-05T01:14:18.911108+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:14:18.911108+0000","last_undegraded":"2024-09-05T01:14:18.911108+0000","last_fullsized":"2024-09-05T01:14:18.911108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833868+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:14:18.833868+0000","last_peered":"2024-09-05T01:14:18.833868+0000","last_clean":"2024-09-05T01:14:18.833868+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:14:18.833868+0000","last_undegraded":"2024-09-05T01:14:18.833868+0000","last_fullsized":"2024-09-05T01:14:18.833868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833028+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:14:18.833028+0000","last_peered":"2024-09-05T01:14:18.833028+0000","last_clean":"2024-09-05T01:14:18.833028+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:14:18.833028+0000","last_undegraded":"2024-09-05T01:14:18.833028+0000","last_fullsized":"2024-09-05T01:14:18.833028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912517+0000","last_change":"2024-09-05T01:14:18.912517+0000","last_active":"2024-09-05T01:14:18.912517+0000","last_peered":"2024-09-05T01:14:18.912517+0000","last_clean":"2024-09-05T01:14:18.912517+0000","last_became_active":"2024-09-05T01:14:18.912074+0000","last_became_peered":"2024-09-05T01:14:18.912074+0000","last_unstale":"2024-09-05T01:14:18.912517+0000","last_undegraded":"2024-09-05T01:14:18.912517+0000","last_fullsized":"2024-09-05T01:14:18.912517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:16.172775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910957+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:14:18.910957+0000","last_peered":"2024-09-05T01:14:18.910957+0000","last_clean":"2024-09-05T01:14:18.910957+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:14:18.910957+0000","last_undegraded":"2024-09-05T01:14:18.910957+0000","last_fullsized":"2024-09-05T01:14:18.910957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:35.925492+0000","last_change":"2024-09-05T01:13:03.885433+0000","last_active":"2024-09-05T01:14:35.925492+0000","last_peered":"2024-09-05T01:14:35.925492+0000","last_clean":"2024-09-05T01:14:35.925492+0000","last_became_active":"2024-09-05T01:13:03.885051+0000","last_became_peered":"2024-09-05T01:13:03.885051+0000","last_unstale":"2024-09-05T01:14:35.925492+0000","last_undegraded":"2024-09-05T01:14:35.925492+0000","last_fullsized":"2024-09-05T01:14:35.925492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:26:56.989141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838947+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:14:18.838947+0000","last_peered":"2024-09-05T01:14:18.838947+0000","last_clean":"2024-09-05T01:14:18.838947+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:14:18.838947+0000","last_undegraded":"2024-09-05T01:14:18.838947+0000","last_fullsized":"2024-09-05T01:14:18.838947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911287+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:14:18.911287+0000","last_peered":"2024-09-05T01:14:18.911287+0000","last_clean":"2024-09-05T01:14:18.911287+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:14:18.911287+0000","last_undegraded":"2024-09-05T01:14:18.911287+0000","last_fullsized":"2024-09-05T01:14:18.911287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834331+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:14:18.834331+0000","last_peered":"2024-09-05T01:14:18.834331+0000","last_clean":"2024-09-05T01:14:18.834331+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:14:18.834331+0000","last_undegraded":"2024-09-05T01:14:18.834331+0000","last_fullsized":"2024-09-05T01:14:18.834331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911176+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:14:18.911176+0000","last_peered":"2024-09-05T01:14:18.911176+0000","last_clean":"2024-09-05T01:14:18.911176+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:14:18.911176+0000","last_undegraded":"2024-09-05T01:14:18.911176+0000","last_fullsized":"2024-09-05T01:14:18.911176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910789+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:14:18.910789+0000","last_peered":"2024-09-05T01:14:18.910789+0000","last_clean":"2024-09-05T01:14:18.910789+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:14:18.910789+0000","last_undegraded":"2024-09-05T01:14:18.910789+0000","last_fullsized":"2024-09-05T01:14:18.910789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834612+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:14:18.834612+0000","last_peered":"2024-09-05T01:14:18.834612+0000","last_clean":"2024-09-05T01:14:18.834612+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:14:18.834612+0000","last_undegraded":"2024-09-05T01:14:18.834612+0000","last_fullsized":"2024-09-05T01:14:18.834612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834389+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:14:18.834389+0000","last_peered":"2024-09-05T01:14:18.834389+0000","last_clean":"2024-09-05T01:14:18.834389+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:14:18.834389+0000","last_undegraded":"2024-09-05T01:14:18.834389+0000","last_fullsized":"2024-09-05T01:14:18.834389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835027+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:14:18.835027+0000","last_peered":"2024-09-05T01:14:18.835027+0000","last_clean":"2024-09-05T01:14:18.835027+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:14:18.835027+0000","last_undegraded":"2024-09-05T01:14:18.835027+0000","last_fullsized":"2024-09-05T01:14:18.835027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911083+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:14:18.911083+0000","last_peered":"2024-09-05T01:14:18.911083+0000","last_clean":"2024-09-05T01:14:18.911083+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:14:18.911083+0000","last_undegraded":"2024-09-05T01:14:18.911083+0000","last_fullsized":"2024-09-05T01:14:18.911083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911008+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:14:18.911008+0000","last_peered":"2024-09-05T01:14:18.911008+0000","last_clean":"2024-09-05T01:14:18.911008+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:14:18.911008+0000","last_undegraded":"2024-09-05T01:14:18.911008+0000","last_fullsized":"2024-09-05T01:14:18.911008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837038+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:14:18.837038+0000","last_peered":"2024-09-05T01:14:18.837038+0000","last_clean":"2024-09-05T01:14:18.837038+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:14:18.837038+0000","last_undegraded":"2024-09-05T01:14:18.837038+0000","last_fullsized":"2024-09-05T01:14:18.837038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910803+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:14:18.910803+0000","last_peered":"2024-09-05T01:14:18.910803+0000","last_clean":"2024-09-05T01:14:18.910803+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:14:18.910803+0000","last_undegraded":"2024-09-05T01:14:18.910803+0000","last_fullsized":"2024-09-05T01:14:18.910803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911444+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:14:18.911444+0000","last_peered":"2024-09-05T01:14:18.911444+0000","last_clean":"2024-09-05T01:14:18.911444+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:14:18.911444+0000","last_undegraded":"2024-09-05T01:14:18.911444+0000","last_fullsized":"2024-09-05T01:14:18.911444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834732+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:14:18.834732+0000","last_peered":"2024-09-05T01:14:18.834732+0000","last_clean":"2024-09-05T01:14:18.834732+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:14:18.834732+0000","last_undegraded":"2024-09-05T01:14:18.834732+0000","last_fullsized":"2024-09-05T01:14:18.834732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840355+0000","last_change":"2024-09-05T01:14:18.840355+0000","last_active":"2024-09-05T01:14:18.840355+0000","last_peered":"2024-09-05T01:14:18.840355+0000","last_clean":"2024-09-05T01:14:18.840355+0000","last_became_active":"2024-09-05T01:14:18.839887+0000","last_became_peered":"2024-09-05T01:14:18.839887+0000","last_unstale":"2024-09-05T01:14:18.840355+0000","last_undegraded":"2024-09-05T01:14:18.840355+0000","last_fullsized":"2024-09-05T01:14:18.840355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:49.152575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911396+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:14:18.911396+0000","last_peered":"2024-09-05T01:14:18.911396+0000","last_clean":"2024-09-05T01:14:18.911396+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:14:18.911396+0000","last_undegraded":"2024-09-05T01:14:18.911396+0000","last_fullsized":"2024-09-05T01:14:18.911396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831789+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:14:18.831789+0000","last_peered":"2024-09-05T01:14:18.831789+0000","last_clean":"2024-09-05T01:14:18.831789+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:14:18.831789+0000","last_undegraded":"2024-09-05T01:14:18.831789+0000","last_fullsized":"2024-09-05T01:14:18.831789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835470+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:14:18.835470+0000","last_peered":"2024-09-05T01:14:18.835470+0000","last_clean":"2024-09-05T01:14:18.835470+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:14:18.835470+0000","last_undegraded":"2024-09-05T01:14:18.835470+0000","last_fullsized":"2024-09-05T01:14:18.835470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911195+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:14:18.911195+0000","last_peered":"2024-09-05T01:14:18.911195+0000","last_clean":"2024-09-05T01:14:18.911195+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:14:18.911195+0000","last_undegraded":"2024-09-05T01:14:18.911195+0000","last_fullsized":"2024-09-05T01:14:18.911195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833879+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:14:18.833879+0000","last_peered":"2024-09-05T01:14:18.833879+0000","last_clean":"2024-09-05T01:14:18.833879+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:14:18.833879+0000","last_undegraded":"2024-09-05T01:14:18.833879+0000","last_fullsized":"2024-09-05T01:14:18.833879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833934+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:14:18.833934+0000","last_peered":"2024-09-05T01:14:18.833934+0000","last_clean":"2024-09-05T01:14:18.833934+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:14:18.833934+0000","last_undegraded":"2024-09-05T01:14:18.833934+0000","last_fullsized":"2024-09-05T01:14:18.833934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836841+0000","last_change":"2024-09-05T01:14:18.836841+0000","last_active":"2024-09-05T01:14:18.836841+0000","last_peered":"2024-09-05T01:14:18.836841+0000","last_clean":"2024-09-05T01:14:18.836841+0000","last_became_active":"2024-09-05T01:14:18.836464+0000","last_became_peered":"2024-09-05T01:14:18.836464+0000","last_unstale":"2024-09-05T01:14:18.836841+0000","last_undegraded":"2024-09-05T01:14:18.836841+0000","last_fullsized":"2024-09-05T01:14:18.836841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:21:02.858269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834220+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:14:18.834220+0000","last_peered":"2024-09-05T01:14:18.834220+0000","last_clean":"2024-09-05T01:14:18.834220+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:14:18.834220+0000","last_undegraded":"2024-09-05T01:14:18.834220+0000","last_fullsized":"2024-09-05T01:14:18.834220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912623+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:14:18.912623+0000","last_peered":"2024-09-05T01:14:18.912623+0000","last_clean":"2024-09-05T01:14:18.912623+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:14:18.912623+0000","last_undegraded":"2024-09-05T01:14:18.912623+0000","last_fullsized":"2024-09-05T01:14:18.912623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833329+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:14:18.833329+0000","last_peered":"2024-09-05T01:14:18.833329+0000","last_clean":"2024-09-05T01:14:18.833329+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:14:18.833329+0000","last_undegraded":"2024-09-05T01:14:18.833329+0000","last_fullsized":"2024-09-05T01:14:18.833329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911174+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:14:18.911174+0000","last_peered":"2024-09-05T01:14:18.911174+0000","last_clean":"2024-09-05T01:14:18.911174+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:14:18.911174+0000","last_undegraded":"2024-09-05T01:14:18.911174+0000","last_fullsized":"2024-09-05T01:14:18.911174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911008+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:14:18.911008+0000","last_peered":"2024-09-05T01:14:18.911008+0000","last_clean":"2024-09-05T01:14:18.911008+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:14:18.911008+0000","last_undegraded":"2024-09-05T01:14:18.911008+0000","last_fullsized":"2024-09-05T01:14:18.911008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911593+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:14:18.911593+0000","last_peered":"2024-09-05T01:14:18.911593+0000","last_clean":"2024-09-05T01:14:18.911593+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:14:18.911593+0000","last_undegraded":"2024-09-05T01:14:18.911593+0000","last_fullsized":"2024-09-05T01:14:18.911593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.832918+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:14:18.832918+0000","last_peered":"2024-09-05T01:14:18.832918+0000","last_clean":"2024-09-05T01:14:18.832918+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:14:18.832918+0000","last_undegraded":"2024-09-05T01:14:18.832918+0000","last_fullsized":"2024-09-05T01:14:18.832918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838715+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:14:18.838715+0000","last_peered":"2024-09-05T01:14:18.838715+0000","last_clean":"2024-09-05T01:14:18.838715+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:14:18.838715+0000","last_undegraded":"2024-09-05T01:14:18.838715+0000","last_fullsized":"2024-09-05T01:14:18.838715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834110+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:14:18.834110+0000","last_peered":"2024-09-05T01:14:18.834110+0000","last_clean":"2024-09-05T01:14:18.834110+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:14:18.834110+0000","last_undegraded":"2024-09-05T01:14:18.834110+0000","last_fullsized":"2024-09-05T01:14:18.834110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834212+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:14:18.834212+0000","last_peered":"2024-09-05T01:14:18.834212+0000","last_clean":"2024-09-05T01:14:18.834212+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:14:18.834212+0000","last_undegraded":"2024-09-05T01:14:18.834212+0000","last_fullsized":"2024-09-05T01:14:18.834212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834721+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:14:18.834721+0000","last_peered":"2024-09-05T01:14:18.834721+0000","last_clean":"2024-09-05T01:14:18.834721+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:14:18.834721+0000","last_undegraded":"2024-09-05T01:14:18.834721+0000","last_fullsized":"2024-09-05T01:14:18.834721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835458+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:14:18.835458+0000","last_peered":"2024-09-05T01:14:18.835458+0000","last_clean":"2024-09-05T01:14:18.835458+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:14:18.835458+0000","last_undegraded":"2024-09-05T01:14:18.835458+0000","last_fullsized":"2024-09-05T01:14:18.835458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":322,"num_read_kb":232,"num_write":167,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":495545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708156,"statfs":{"total":95995035648,"available":95957151744,"internally_reserved":0,"allocated":1687552,"data_stored":1144889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972394,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":499877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201327,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1015808,"data_stored":495701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1609728,"data_stored":1091612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":995328,"data_stored":490957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822718,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1617920,"data_stored":1087200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986244,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":495607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215176,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":554686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1060864,"data_stored":552943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":492345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:14:44.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:14:44.625 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:44.931 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:44 smithi129 ceph-mon[30979]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:44.932 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:44 smithi129 ceph-mon[30979]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:45.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 -- 172.21.15.129:0/1497795923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 msgr2=0x7fb88c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:45.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 --2- 172.21.15.129:0/1497795923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c10e610 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb884009ca0 tx=0x7fb88402a950 comp rx=0 tx=0).stop 2024-09-05T01:14:45.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 -- 172.21.15.129:0/1497795923 shutdown_connections 2024-09-05T01:14:45.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 --2- 172.21.15.129:0/1497795923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c10e610 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 --2- 172.21.15.129:0/1497795923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 --2- 172.21.15.129:0/1497795923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 -- 172.21.15.129:0/1497795923 >> 172.21.15.129:0/1497795923 conn(0x7fb88c0fc7a0 msgr2=0x7fb88c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 -- 172.21.15.129:0/1497795923 shutdown_connections 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.124+0000 7fb89280b640 1 -- 172.21.15.129:0/1497795923 wait complete. 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 Processor -- start 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 -- start start 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c111e00 con 0x7fb88c10c170 2024-09-05T01:14:45.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c111c80 con 0x7fb88c102d40 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb89280b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb88c111f80 con 0x7fb88c103660 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb890da8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb890da8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47072/0 (socket says 172.21.15.129:47072) 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.125+0000 7fb890da8640 1 -- 172.21.15.129:0/1388868622 learned_addr learned my addr 172.21.15.129:0/1388868622 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb88bfff640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 -- 172.21.15.129:0/1388868622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 msgr2=0x7fb88c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb88b7fe640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 -- 172.21.15.129:0/1388868622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 msgr2=0x7fb88c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 -- 172.21.15.129:0/1388868622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88c19d2f0 con 0x7fb88c10c170 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb88bfff640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb88b7fe640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:45.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb890da8640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb88402ae70 tx=0x7fb88402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884034040 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8840308f0 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88c19d580 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840047f0 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.126+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb88c1a4ec0 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.127+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb88c103120 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.128+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb884037020 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.129+0000 7fb8897fa640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 0x7fb8740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.129+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb8840f04e0 con 0x7fb88c10c170 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.129+0000 7fb88bfff640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 0x7fb8740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.129+0000 7fb88bfff640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 0x7fb8740a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005d00 tx=0x7fb87c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:45.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.133+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb8840bd090 con 0x7fb88c10c170 2024-09-05T01:14:45.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:44 smithi183 ceph-mon[34589]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:45.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:44 smithi183 ceph-mon[34589]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:44 smithi194 ceph-mon[34941]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:44 smithi194 ceph-mon[34941]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:45.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.294+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb88c062630 con 0x7fb8740a2c10 2024-09-05T01:14:45.304 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:45.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.302+0000 7fb8897fa640 1 -- 172.21.15.129:0/1388868622 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340344 (secure 0 0 0) 0x7fb88c062630 con 0x7fb8740a2c10 2024-09-05T01:14:45.308 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:14:45.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 msgr2=0x7fb8740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 0x7fb8740a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb87c005d00 tx=0x7fb87c009550 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 msgr2=0x7fb88c19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb88402ae70 tx=0x7fb88402f5d0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 shutdown_connections 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8740a2c10 0x7fb8740a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb88c10c170 0x7fb88c19cc10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb88c103660 0x7fb88c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 --2- 172.21.15.129:0/1388868622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb88c102d40 0x7fb88c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 >> 172.21.15.129:0/1388868622 conn(0x7fb88c0fc7a0 msgr2=0x7fb88c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 shutdown_connections 2024-09-05T01:14:45.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:45.307+0000 7fb89280b640 1 -- 172.21.15.129:0/1388868622 wait complete. 2024-09-05T01:14:46.006 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-05T01:14:43.771684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":478,"num_read_kb":356,"num_write":350,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392248,"kb_used_data":13796,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549576,"statfs":{"total":1151940427776,"available":1151538765824,"internally_reserved":0,"allocated":14127104,"data_stored":7892513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003436"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831970+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:14:18.831970+0000","last_peered":"2024-09-05T01:14:18.831970+0000","last_clean":"2024-09-05T01:14:18.831970+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:14:18.831970+0000","last_undegraded":"2024-09-05T01:14:18.831970+0000","last_fullsized":"2024-09-05T01:14:18.831970+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911065+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:14:18.911065+0000","last_peered":"2024-09-05T01:14:18.911065+0000","last_clean":"2024-09-05T01:14:18.911065+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:14:18.911065+0000","last_undegraded":"2024-09-05T01:14:18.911065+0000","last_fullsized":"2024-09-05T01:14:18.911065+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910723+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:14:18.910723+0000","last_peered":"2024-09-05T01:14:18.910723+0000","last_clean":"2024-09-05T01:14:18.910723+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:14:18.910723+0000","last_undegraded":"2024-09-05T01:14:18.910723+0000","last_fullsized":"2024-09-05T01:14:18.910723+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911282+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:14:18.911282+0000","last_peered":"2024-09-05T01:14:18.911282+0000","last_clean":"2024-09-05T01:14:18.911282+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:14:18.911282+0000","last_undegraded":"2024-09-05T01:14:18.911282+0000","last_fullsized":"2024-09-05T01:14:18.911282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834737+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:14:18.834737+0000","last_peered":"2024-09-05T01:14:18.834737+0000","last_clean":"2024-09-05T01:14:18.834737+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:14:18.834737+0000","last_undegraded":"2024-09-05T01:14:18.834737+0000","last_fullsized":"2024-09-05T01:14:18.834737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913937+0000","last_change":"2024-09-05T01:14:18.913937+0000","last_active":"2024-09-05T01:14:18.913937+0000","last_peered":"2024-09-05T01:14:18.913937+0000","last_clean":"2024-09-05T01:14:18.913937+0000","last_became_active":"2024-09-05T01:14:18.913425+0000","last_became_peered":"2024-09-05T01:14:18.913425+0000","last_unstale":"2024-09-05T01:14:18.913937+0000","last_undegraded":"2024-09-05T01:14:18.913937+0000","last_fullsized":"2024-09-05T01:14:18.913937+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:56.174787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839094+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:14:18.839094+0000","last_peered":"2024-09-05T01:14:18.839094+0000","last_clean":"2024-09-05T01:14:18.839094+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:14:18.839094+0000","last_undegraded":"2024-09-05T01:14:18.839094+0000","last_fullsized":"2024-09-05T01:14:18.839094+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911672+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:14:18.911672+0000","last_peered":"2024-09-05T01:14:18.911672+0000","last_clean":"2024-09-05T01:14:18.911672+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:14:18.911672+0000","last_undegraded":"2024-09-05T01:14:18.911672+0000","last_fullsized":"2024-09-05T01:14:18.911672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833934+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:14:18.833934+0000","last_peered":"2024-09-05T01:14:18.833934+0000","last_clean":"2024-09-05T01:14:18.833934+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:14:18.833934+0000","last_undegraded":"2024-09-05T01:14:18.833934+0000","last_fullsized":"2024-09-05T01:14:18.833934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839216+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:14:18.839216+0000","last_peered":"2024-09-05T01:14:18.839216+0000","last_clean":"2024-09-05T01:14:18.839216+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:14:18.839216+0000","last_undegraded":"2024-09-05T01:14:18.839216+0000","last_fullsized":"2024-09-05T01:14:18.839216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913351+0000","last_change":"2024-09-05T01:14:18.913351+0000","last_active":"2024-09-05T01:14:18.913351+0000","last_peered":"2024-09-05T01:14:18.913351+0000","last_clean":"2024-09-05T01:14:18.913351+0000","last_became_active":"2024-09-05T01:14:18.912935+0000","last_became_peered":"2024-09-05T01:14:18.912935+0000","last_unstale":"2024-09-05T01:14:18.913351+0000","last_undegraded":"2024-09-05T01:14:18.913351+0000","last_fullsized":"2024-09-05T01:14:18.913351+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:32:30.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833180+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:14:18.833180+0000","last_peered":"2024-09-05T01:14:18.833180+0000","last_clean":"2024-09-05T01:14:18.833180+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:14:18.833180+0000","last_undegraded":"2024-09-05T01:14:18.833180+0000","last_fullsized":"2024-09-05T01:14:18.833180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912209+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:14:18.912209+0000","last_peered":"2024-09-05T01:14:18.912209+0000","last_clean":"2024-09-05T01:14:18.912209+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:14:18.912209+0000","last_undegraded":"2024-09-05T01:14:18.912209+0000","last_fullsized":"2024-09-05T01:14:18.912209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838781+0000","last_change":"2024-09-05T01:13:03.882357+0000","last_active":"2024-09-05T01:14:18.838781+0000","last_peered":"2024-09-05T01:14:18.838781+0000","last_clean":"2024-09-05T01:14:18.838781+0000","last_became_active":"2024-09-05T01:13:03.881983+0000","last_became_peered":"2024-09-05T01:13:03.881983+0000","last_unstale":"2024-09-05T01:14:18.838781+0000","last_undegraded":"2024-09-05T01:14:18.838781+0000","last_fullsized":"2024-09-05T01:14:18.838781+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:09:44.521251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833493+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:14:18.833493+0000","last_peered":"2024-09-05T01:14:18.833493+0000","last_clean":"2024-09-05T01:14:18.833493+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:14:18.833493+0000","last_undegraded":"2024-09-05T01:14:18.833493+0000","last_fullsized":"2024-09-05T01:14:18.833493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911268+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:14:18.911268+0000","last_peered":"2024-09-05T01:14:18.911268+0000","last_clean":"2024-09-05T01:14:18.911268+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:14:18.911268+0000","last_undegraded":"2024-09-05T01:14:18.911268+0000","last_fullsized":"2024-09-05T01:14:18.911268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910837+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:14:18.910837+0000","last_peered":"2024-09-05T01:14:18.910837+0000","last_clean":"2024-09-05T01:14:18.910837+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:14:18.910837+0000","last_undegraded":"2024-09-05T01:14:18.910837+0000","last_fullsized":"2024-09-05T01:14:18.910837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833763+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:14:18.833763+0000","last_peered":"2024-09-05T01:14:18.833763+0000","last_clean":"2024-09-05T01:14:18.833763+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:14:18.833763+0000","last_undegraded":"2024-09-05T01:14:18.833763+0000","last_fullsized":"2024-09-05T01:14:18.833763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834435+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:14:18.834435+0000","last_peered":"2024-09-05T01:14:18.834435+0000","last_clean":"2024-09-05T01:14:18.834435+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:14:18.834435+0000","last_undegraded":"2024-09-05T01:14:18.834435+0000","last_fullsized":"2024-09-05T01:14:18.834435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911382+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:14:18.911382+0000","last_peered":"2024-09-05T01:14:18.911382+0000","last_clean":"2024-09-05T01:14:18.911382+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:14:18.911382+0000","last_undegraded":"2024-09-05T01:14:18.911382+0000","last_fullsized":"2024-09-05T01:14:18.911382+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831810+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:14:18.831810+0000","last_peered":"2024-09-05T01:14:18.831810+0000","last_clean":"2024-09-05T01:14:18.831810+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:14:18.831810+0000","last_undegraded":"2024-09-05T01:14:18.831810+0000","last_fullsized":"2024-09-05T01:14:18.831810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838595+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:14:18.838595+0000","last_peered":"2024-09-05T01:14:18.838595+0000","last_clean":"2024-09-05T01:14:18.838595+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:14:18.838595+0000","last_undegraded":"2024-09-05T01:14:18.838595+0000","last_fullsized":"2024-09-05T01:14:18.838595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911346+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:14:18.911346+0000","last_peered":"2024-09-05T01:14:18.911346+0000","last_clean":"2024-09-05T01:14:18.911346+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:14:18.911346+0000","last_undegraded":"2024-09-05T01:14:18.911346+0000","last_fullsized":"2024-09-05T01:14:18.911346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833601+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:14:18.833601+0000","last_peered":"2024-09-05T01:14:18.833601+0000","last_clean":"2024-09-05T01:14:18.833601+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:14:18.833601+0000","last_undegraded":"2024-09-05T01:14:18.833601+0000","last_fullsized":"2024-09-05T01:14:18.833601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912689+0000","last_change":"2024-09-05T01:14:18.912689+0000","last_active":"2024-09-05T01:14:18.912689+0000","last_peered":"2024-09-05T01:14:18.912689+0000","last_clean":"2024-09-05T01:14:18.912689+0000","last_became_active":"2024-09-05T01:14:18.912358+0000","last_became_peered":"2024-09-05T01:14:18.912358+0000","last_unstale":"2024-09-05T01:14:18.912689+0000","last_undegraded":"2024-09-05T01:14:18.912689+0000","last_fullsized":"2024-09-05T01:14:18.912689+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:59.876348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834488+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:14:18.834488+0000","last_peered":"2024-09-05T01:14:18.834488+0000","last_clean":"2024-09-05T01:14:18.834488+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:14:18.834488+0000","last_undegraded":"2024-09-05T01:14:18.834488+0000","last_fullsized":"2024-09-05T01:14:18.834488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910736+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:14:18.910736+0000","last_peered":"2024-09-05T01:14:18.910736+0000","last_clean":"2024-09-05T01:14:18.910736+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:14:18.910736+0000","last_undegraded":"2024-09-05T01:14:18.910736+0000","last_fullsized":"2024-09-05T01:14:18.910736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911051+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:14:18.911051+0000","last_peered":"2024-09-05T01:14:18.911051+0000","last_clean":"2024-09-05T01:14:18.911051+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:14:18.911051+0000","last_undegraded":"2024-09-05T01:14:18.911051+0000","last_fullsized":"2024-09-05T01:14:18.911051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831875+0000","last_change":"2024-09-05T01:13:03.873898+0000","last_active":"2024-09-05T01:14:18.831875+0000","last_peered":"2024-09-05T01:14:18.831875+0000","last_clean":"2024-09-05T01:14:18.831875+0000","last_became_active":"2024-09-05T01:13:03.873546+0000","last_became_peered":"2024-09-05T01:13:03.873546+0000","last_unstale":"2024-09-05T01:14:18.831875+0000","last_undegraded":"2024-09-05T01:14:18.831875+0000","last_fullsized":"2024-09-05T01:14:18.831875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:45:48.842042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910909+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:14:18.910909+0000","last_peered":"2024-09-05T01:14:18.910909+0000","last_clean":"2024-09-05T01:14:18.910909+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:14:18.910909+0000","last_undegraded":"2024-09-05T01:14:18.910909+0000","last_fullsized":"2024-09-05T01:14:18.910909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834477+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:14:18.834477+0000","last_peered":"2024-09-05T01:14:18.834477+0000","last_clean":"2024-09-05T01:14:18.834477+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:14:18.834477+0000","last_undegraded":"2024-09-05T01:14:18.834477+0000","last_fullsized":"2024-09-05T01:14:18.834477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.839192+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:14:18.839192+0000","last_peered":"2024-09-05T01:14:18.839192+0000","last_clean":"2024-09-05T01:14:18.839192+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:14:18.839192+0000","last_undegraded":"2024-09-05T01:14:18.839192+0000","last_fullsized":"2024-09-05T01:14:18.839192+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834417+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:14:18.834417+0000","last_peered":"2024-09-05T01:14:18.834417+0000","last_clean":"2024-09-05T01:14:18.834417+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:14:18.834417+0000","last_undegraded":"2024-09-05T01:14:18.834417+0000","last_fullsized":"2024-09-05T01:14:18.834417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833148+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:14:18.833148+0000","last_peered":"2024-09-05T01:14:18.833148+0000","last_clean":"2024-09-05T01:14:18.833148+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:14:18.833148+0000","last_undegraded":"2024-09-05T01:14:18.833148+0000","last_fullsized":"2024-09-05T01:14:18.833148+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831415+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:14:18.831415+0000","last_peered":"2024-09-05T01:14:18.831415+0000","last_clean":"2024-09-05T01:14:18.831415+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:14:18.831415+0000","last_undegraded":"2024-09-05T01:14:18.831415+0000","last_fullsized":"2024-09-05T01:14:18.831415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838813+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:14:18.838813+0000","last_peered":"2024-09-05T01:14:18.838813+0000","last_clean":"2024-09-05T01:14:18.838813+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:14:18.838813+0000","last_undegraded":"2024-09-05T01:14:18.838813+0000","last_fullsized":"2024-09-05T01:14:18.838813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.832774+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:14:18.832774+0000","last_peered":"2024-09-05T01:14:18.832774+0000","last_clean":"2024-09-05T01:14:18.832774+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:14:18.832774+0000","last_undegraded":"2024-09-05T01:14:18.832774+0000","last_fullsized":"2024-09-05T01:14:18.832774+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912415+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:14:18.912415+0000","last_peered":"2024-09-05T01:14:18.912415+0000","last_clean":"2024-09-05T01:14:18.912415+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:14:18.912415+0000","last_undegraded":"2024-09-05T01:14:18.912415+0000","last_fullsized":"2024-09-05T01:14:18.912415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834243+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:14:18.834243+0000","last_peered":"2024-09-05T01:14:18.834243+0000","last_clean":"2024-09-05T01:14:18.834243+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:14:18.834243+0000","last_undegraded":"2024-09-05T01:14:18.834243+0000","last_fullsized":"2024-09-05T01:14:18.834243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912776+0000","last_change":"2024-09-05T01:14:18.912776+0000","last_active":"2024-09-05T01:14:18.912776+0000","last_peered":"2024-09-05T01:14:18.912776+0000","last_clean":"2024-09-05T01:14:18.912776+0000","last_became_active":"2024-09-05T01:14:18.912337+0000","last_became_peered":"2024-09-05T01:14:18.912337+0000","last_unstale":"2024-09-05T01:14:18.912776+0000","last_undegraded":"2024-09-05T01:14:18.912776+0000","last_fullsized":"2024-09-05T01:14:18.912776+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:37:57.935580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:41.932830+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:14:41.932830+0000","last_peered":"2024-09-05T01:14:41.932830+0000","last_clean":"2024-09-05T01:14:41.932830+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:14:41.932830+0000","last_undegraded":"2024-09-05T01:14:41.932830+0000","last_fullsized":"2024-09-05T01:14:41.932830+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834288+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:14:18.834288+0000","last_peered":"2024-09-05T01:14:18.834288+0000","last_clean":"2024-09-05T01:14:18.834288+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:14:18.834288+0000","last_undegraded":"2024-09-05T01:14:18.834288+0000","last_fullsized":"2024-09-05T01:14:18.834288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833762+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:14:18.833762+0000","last_peered":"2024-09-05T01:14:18.833762+0000","last_clean":"2024-09-05T01:14:18.833762+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:14:18.833762+0000","last_undegraded":"2024-09-05T01:14:18.833762+0000","last_fullsized":"2024-09-05T01:14:18.833762+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911783+0000","last_change":"2024-09-05T01:13:03.880884+0000","last_active":"2024-09-05T01:14:18.911783+0000","last_peered":"2024-09-05T01:14:18.911783+0000","last_clean":"2024-09-05T01:14:18.911783+0000","last_became_active":"2024-09-05T01:13:03.880274+0000","last_became_peered":"2024-09-05T01:13:03.880274+0000","last_unstale":"2024-09-05T01:14:18.911783+0000","last_undegraded":"2024-09-05T01:14:18.911783+0000","last_fullsized":"2024-09-05T01:14:18.911783+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:24.973934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836621+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:14:18.836621+0000","last_peered":"2024-09-05T01:14:18.836621+0000","last_clean":"2024-09-05T01:14:18.836621+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:14:18.836621+0000","last_undegraded":"2024-09-05T01:14:18.836621+0000","last_fullsized":"2024-09-05T01:14:18.836621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910805+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:14:18.910805+0000","last_peered":"2024-09-05T01:14:18.910805+0000","last_clean":"2024-09-05T01:14:18.910805+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:14:18.910805+0000","last_undegraded":"2024-09-05T01:14:18.910805+0000","last_fullsized":"2024-09-05T01:14:18.910805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834236+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:14:18.834236+0000","last_peered":"2024-09-05T01:14:18.834236+0000","last_clean":"2024-09-05T01:14:18.834236+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:14:18.834236+0000","last_undegraded":"2024-09-05T01:14:18.834236+0000","last_fullsized":"2024-09-05T01:14:18.834236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911981+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:14:18.911981+0000","last_peered":"2024-09-05T01:14:18.911981+0000","last_clean":"2024-09-05T01:14:18.911981+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:14:18.911981+0000","last_undegraded":"2024-09-05T01:14:18.911981+0000","last_fullsized":"2024-09-05T01:14:18.911981+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912424+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:14:18.912424+0000","last_peered":"2024-09-05T01:14:18.912424+0000","last_clean":"2024-09-05T01:14:18.912424+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:14:18.912424+0000","last_undegraded":"2024-09-05T01:14:18.912424+0000","last_fullsized":"2024-09-05T01:14:18.912424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912677+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:14:18.912677+0000","last_peered":"2024-09-05T01:14:18.912677+0000","last_clean":"2024-09-05T01:14:18.912677+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:14:18.912677+0000","last_undegraded":"2024-09-05T01:14:18.912677+0000","last_fullsized":"2024-09-05T01:14:18.912677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911573+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:14:18.911573+0000","last_peered":"2024-09-05T01:14:18.911573+0000","last_clean":"2024-09-05T01:14:18.911573+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:14:18.911573+0000","last_undegraded":"2024-09-05T01:14:18.911573+0000","last_fullsized":"2024-09-05T01:14:18.911573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910751+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:14:18.910751+0000","last_peered":"2024-09-05T01:14:18.910751+0000","last_clean":"2024-09-05T01:14:18.910751+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:14:18.910751+0000","last_undegraded":"2024-09-05T01:14:18.910751+0000","last_fullsized":"2024-09-05T01:14:18.910751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912624+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:14:18.912624+0000","last_peered":"2024-09-05T01:14:18.912624+0000","last_clean":"2024-09-05T01:14:18.912624+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:14:18.912624+0000","last_undegraded":"2024-09-05T01:14:18.912624+0000","last_fullsized":"2024-09-05T01:14:18.912624+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837100+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:14:18.837100+0000","last_peered":"2024-09-05T01:14:18.837100+0000","last_clean":"2024-09-05T01:14:18.837100+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:14:18.837100+0000","last_undegraded":"2024-09-05T01:14:18.837100+0000","last_fullsized":"2024-09-05T01:14:18.837100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834531+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:14:18.834531+0000","last_peered":"2024-09-05T01:14:18.834531+0000","last_clean":"2024-09-05T01:14:18.834531+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:14:18.834531+0000","last_undegraded":"2024-09-05T01:14:18.834531+0000","last_fullsized":"2024-09-05T01:14:18.834531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912761+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:14:18.912761+0000","last_peered":"2024-09-05T01:14:18.912761+0000","last_clean":"2024-09-05T01:14:18.912761+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:14:18.912761+0000","last_undegraded":"2024-09-05T01:14:18.912761+0000","last_fullsized":"2024-09-05T01:14:18.912761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834786+0000","last_change":"2024-09-05T01:13:03.877073+0000","last_active":"2024-09-05T01:14:18.834786+0000","last_peered":"2024-09-05T01:14:18.834786+0000","last_clean":"2024-09-05T01:14:18.834786+0000","last_became_active":"2024-09-05T01:13:03.876801+0000","last_became_peered":"2024-09-05T01:13:03.876801+0000","last_unstale":"2024-09-05T01:14:18.834786+0000","last_undegraded":"2024-09-05T01:14:18.834786+0000","last_fullsized":"2024-09-05T01:14:18.834786+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:13:24.959411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.914428+0000","last_change":"2024-09-05T01:14:18.913738+0000","last_active":"2024-09-05T01:14:18.914428+0000","last_peered":"2024-09-05T01:14:18.914428+0000","last_clean":"2024-09-05T01:14:18.914428+0000","last_became_active":"2024-09-05T01:14:18.913084+0000","last_became_peered":"2024-09-05T01:14:18.913084+0000","last_unstale":"2024-09-05T01:14:18.914428+0000","last_undegraded":"2024-09-05T01:14:18.914428+0000","last_fullsized":"2024-09-05T01:14:18.914428+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:43.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835189+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:14:18.835189+0000","last_peered":"2024-09-05T01:14:18.835189+0000","last_clean":"2024-09-05T01:14:18.835189+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:14:18.835189+0000","last_undegraded":"2024-09-05T01:14:18.835189+0000","last_fullsized":"2024-09-05T01:14:18.835189+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838959+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:14:18.838959+0000","last_peered":"2024-09-05T01:14:18.838959+0000","last_clean":"2024-09-05T01:14:18.838959+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:14:18.838959+0000","last_undegraded":"2024-09-05T01:14:18.838959+0000","last_fullsized":"2024-09-05T01:14:18.838959+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836008+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:14:18.836008+0000","last_peered":"2024-09-05T01:14:18.836008+0000","last_clean":"2024-09-05T01:14:18.836008+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:14:18.836008+0000","last_undegraded":"2024-09-05T01:14:18.836008+0000","last_fullsized":"2024-09-05T01:14:18.836008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836188+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:14:18.836188+0000","last_peered":"2024-09-05T01:14:18.836188+0000","last_clean":"2024-09-05T01:14:18.836188+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:14:18.836188+0000","last_undegraded":"2024-09-05T01:14:18.836188+0000","last_fullsized":"2024-09-05T01:14:18.836188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840016+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:14:18.840016+0000","last_peered":"2024-09-05T01:14:18.840016+0000","last_clean":"2024-09-05T01:14:18.840016+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:14:18.840016+0000","last_undegraded":"2024-09-05T01:14:18.840016+0000","last_fullsized":"2024-09-05T01:14:18.840016+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834690+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:14:18.834690+0000","last_peered":"2024-09-05T01:14:18.834690+0000","last_clean":"2024-09-05T01:14:18.834690+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:14:18.834690+0000","last_undegraded":"2024-09-05T01:14:18.834690+0000","last_fullsized":"2024-09-05T01:14:18.834690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911622+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:14:18.911622+0000","last_peered":"2024-09-05T01:14:18.911622+0000","last_clean":"2024-09-05T01:14:18.911622+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:14:18.911622+0000","last_undegraded":"2024-09-05T01:14:18.911622+0000","last_fullsized":"2024-09-05T01:14:18.911622+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:39.930898+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:14:39.930898+0000","last_peered":"2024-09-05T01:14:39.930898+0000","last_clean":"2024-09-05T01:14:39.930898+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:14:39.930898+0000","last_undegraded":"2024-09-05T01:14:39.930898+0000","last_fullsized":"2024-09-05T01:14:39.930898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831873+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:14:18.831873+0000","last_peered":"2024-09-05T01:14:18.831873+0000","last_clean":"2024-09-05T01:14:18.831873+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:14:18.831873+0000","last_undegraded":"2024-09-05T01:14:18.831873+0000","last_fullsized":"2024-09-05T01:14:18.831873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912320+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:14:18.912320+0000","last_peered":"2024-09-05T01:14:18.912320+0000","last_clean":"2024-09-05T01:14:18.912320+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:14:18.912320+0000","last_undegraded":"2024-09-05T01:14:18.912320+0000","last_fullsized":"2024-09-05T01:14:18.912320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837195+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:14:18.837195+0000","last_peered":"2024-09-05T01:14:18.837195+0000","last_clean":"2024-09-05T01:14:18.837195+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:14:18.837195+0000","last_undegraded":"2024-09-05T01:14:18.837195+0000","last_fullsized":"2024-09-05T01:14:18.837195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834794+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:14:18.834794+0000","last_peered":"2024-09-05T01:14:18.834794+0000","last_clean":"2024-09-05T01:14:18.834794+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:14:18.834794+0000","last_undegraded":"2024-09-05T01:14:18.834794+0000","last_fullsized":"2024-09-05T01:14:18.834794+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834032+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:14:18.834032+0000","last_peered":"2024-09-05T01:14:18.834032+0000","last_clean":"2024-09-05T01:14:18.834032+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:14:18.834032+0000","last_undegraded":"2024-09-05T01:14:18.834032+0000","last_fullsized":"2024-09-05T01:14:18.834032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911479+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:14:18.911479+0000","last_peered":"2024-09-05T01:14:18.911479+0000","last_clean":"2024-09-05T01:14:18.911479+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:14:18.911479+0000","last_undegraded":"2024-09-05T01:14:18.911479+0000","last_fullsized":"2024-09-05T01:14:18.911479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836716+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:14:18.836716+0000","last_peered":"2024-09-05T01:14:18.836716+0000","last_clean":"2024-09-05T01:14:18.836716+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:14:18.836716+0000","last_undegraded":"2024-09-05T01:14:18.836716+0000","last_fullsized":"2024-09-05T01:14:18.836716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838961+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:14:18.838961+0000","last_peered":"2024-09-05T01:14:18.838961+0000","last_clean":"2024-09-05T01:14:18.838961+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:14:18.838961+0000","last_undegraded":"2024-09-05T01:14:18.838961+0000","last_fullsized":"2024-09-05T01:14:18.838961+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'32","reported_seq":183,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:41.933440+0000","last_change":"2024-09-05T01:13:03.883605+0000","last_active":"2024-09-05T01:14:41.933440+0000","last_peered":"2024-09-05T01:14:41.933440+0000","last_clean":"2024-09-05T01:14:41.933440+0000","last_became_active":"2024-09-05T01:13:03.883275+0000","last_became_peered":"2024-09-05T01:13:03.883275+0000","last_unstale":"2024-09-05T01:14:41.933440+0000","last_undegraded":"2024-09-05T01:14:41.933440+0000","last_fullsized":"2024-09-05T01:14:41.933440+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:58:39.763930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":117,"num_read_kb":119,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840367+0000","last_change":"2024-09-05T01:14:18.840367+0000","last_active":"2024-09-05T01:14:18.840367+0000","last_peered":"2024-09-05T01:14:18.840367+0000","last_clean":"2024-09-05T01:14:18.840367+0000","last_became_active":"2024-09-05T01:14:18.839937+0000","last_became_peered":"2024-09-05T01:14:18.839937+0000","last_unstale":"2024-09-05T01:14:18.840367+0000","last_undegraded":"2024-09-05T01:14:18.840367+0000","last_fullsized":"2024-09-05T01:14:18.840367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:49:14.286631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911766+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:14:18.911766+0000","last_peered":"2024-09-05T01:14:18.911766+0000","last_clean":"2024-09-05T01:14:18.911766+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:14:18.911766+0000","last_undegraded":"2024-09-05T01:14:18.911766+0000","last_fullsized":"2024-09-05T01:14:18.911766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836975+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:14:18.836975+0000","last_peered":"2024-09-05T01:14:18.836975+0000","last_clean":"2024-09-05T01:14:18.836975+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:14:18.836975+0000","last_undegraded":"2024-09-05T01:14:18.836975+0000","last_fullsized":"2024-09-05T01:14:18.836975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.913011+0000","last_change":"2024-09-05T01:14:18.913011+0000","last_active":"2024-09-05T01:14:18.913011+0000","last_peered":"2024-09-05T01:14:18.913011+0000","last_clean":"2024-09-05T01:14:18.913011+0000","last_became_active":"2024-09-05T01:14:18.912546+0000","last_became_peered":"2024-09-05T01:14:18.912546+0000","last_unstale":"2024-09-05T01:14:18.913011+0000","last_undegraded":"2024-09-05T01:14:18.913011+0000","last_fullsized":"2024-09-05T01:14:18.913011+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:11:02.202754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834826+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:14:18.834826+0000","last_peered":"2024-09-05T01:14:18.834826+0000","last_clean":"2024-09-05T01:14:18.834826+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:14:18.834826+0000","last_undegraded":"2024-09-05T01:14:18.834826+0000","last_fullsized":"2024-09-05T01:14:18.834826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912511+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:14:18.912511+0000","last_peered":"2024-09-05T01:14:18.912511+0000","last_clean":"2024-09-05T01:14:18.912511+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:14:18.912511+0000","last_undegraded":"2024-09-05T01:14:18.912511+0000","last_fullsized":"2024-09-05T01:14:18.912511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.830938+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:14:18.830938+0000","last_peered":"2024-09-05T01:14:18.830938+0000","last_clean":"2024-09-05T01:14:18.830938+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:14:18.830938+0000","last_undegraded":"2024-09-05T01:14:18.830938+0000","last_fullsized":"2024-09-05T01:14:18.830938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911089+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:14:18.911089+0000","last_peered":"2024-09-05T01:14:18.911089+0000","last_clean":"2024-09-05T01:14:18.911089+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:14:18.911089+0000","last_undegraded":"2024-09-05T01:14:18.911089+0000","last_fullsized":"2024-09-05T01:14:18.911089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":91,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:40.932131+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:14:40.932131+0000","last_peered":"2024-09-05T01:14:40.932131+0000","last_clean":"2024-09-05T01:14:40.932131+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:14:40.932131+0000","last_undegraded":"2024-09-05T01:14:40.932131+0000","last_fullsized":"2024-09-05T01:14:40.932131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910667+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:14:18.910667+0000","last_peered":"2024-09-05T01:14:18.910667+0000","last_clean":"2024-09-05T01:14:18.910667+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:14:18.910667+0000","last_undegraded":"2024-09-05T01:14:18.910667+0000","last_fullsized":"2024-09-05T01:14:18.910667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834661+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:14:18.834661+0000","last_peered":"2024-09-05T01:14:18.834661+0000","last_clean":"2024-09-05T01:14:18.834661+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:14:18.834661+0000","last_undegraded":"2024-09-05T01:14:18.834661+0000","last_fullsized":"2024-09-05T01:14:18.834661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831261+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:14:18.831261+0000","last_peered":"2024-09-05T01:14:18.831261+0000","last_clean":"2024-09-05T01:14:18.831261+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:14:18.831261+0000","last_undegraded":"2024-09-05T01:14:18.831261+0000","last_fullsized":"2024-09-05T01:14:18.831261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837140+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:14:18.837140+0000","last_peered":"2024-09-05T01:14:18.837140+0000","last_clean":"2024-09-05T01:14:18.837140+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:14:18.837140+0000","last_undegraded":"2024-09-05T01:14:18.837140+0000","last_fullsized":"2024-09-05T01:14:18.837140+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835239+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:14:18.835239+0000","last_peered":"2024-09-05T01:14:18.835239+0000","last_clean":"2024-09-05T01:14:18.835239+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:14:18.835239+0000","last_undegraded":"2024-09-05T01:14:18.835239+0000","last_fullsized":"2024-09-05T01:14:18.835239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911108+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:14:18.911108+0000","last_peered":"2024-09-05T01:14:18.911108+0000","last_clean":"2024-09-05T01:14:18.911108+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:14:18.911108+0000","last_undegraded":"2024-09-05T01:14:18.911108+0000","last_fullsized":"2024-09-05T01:14:18.911108+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833868+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:14:18.833868+0000","last_peered":"2024-09-05T01:14:18.833868+0000","last_clean":"2024-09-05T01:14:18.833868+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:14:18.833868+0000","last_undegraded":"2024-09-05T01:14:18.833868+0000","last_fullsized":"2024-09-05T01:14:18.833868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833028+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:14:18.833028+0000","last_peered":"2024-09-05T01:14:18.833028+0000","last_clean":"2024-09-05T01:14:18.833028+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:14:18.833028+0000","last_undegraded":"2024-09-05T01:14:18.833028+0000","last_fullsized":"2024-09-05T01:14:18.833028+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912517+0000","last_change":"2024-09-05T01:14:18.912517+0000","last_active":"2024-09-05T01:14:18.912517+0000","last_peered":"2024-09-05T01:14:18.912517+0000","last_clean":"2024-09-05T01:14:18.912517+0000","last_became_active":"2024-09-05T01:14:18.912074+0000","last_became_peered":"2024-09-05T01:14:18.912074+0000","last_unstale":"2024-09-05T01:14:18.912517+0000","last_undegraded":"2024-09-05T01:14:18.912517+0000","last_fullsized":"2024-09-05T01:14:18.912517+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:16.172775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910957+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:14:18.910957+0000","last_peered":"2024-09-05T01:14:18.910957+0000","last_clean":"2024-09-05T01:14:18.910957+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:14:18.910957+0000","last_undegraded":"2024-09-05T01:14:18.910957+0000","last_fullsized":"2024-09-05T01:14:18.910957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:40.932600+0000","last_change":"2024-09-05T01:13:03.885433+0000","last_active":"2024-09-05T01:14:40.932600+0000","last_peered":"2024-09-05T01:14:40.932600+0000","last_clean":"2024-09-05T01:14:40.932600+0000","last_became_active":"2024-09-05T01:13:03.885051+0000","last_became_peered":"2024-09-05T01:13:03.885051+0000","last_unstale":"2024-09-05T01:14:40.932600+0000","last_undegraded":"2024-09-05T01:14:40.932600+0000","last_fullsized":"2024-09-05T01:14:40.932600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:26:56.989141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838947+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:14:18.838947+0000","last_peered":"2024-09-05T01:14:18.838947+0000","last_clean":"2024-09-05T01:14:18.838947+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:14:18.838947+0000","last_undegraded":"2024-09-05T01:14:18.838947+0000","last_fullsized":"2024-09-05T01:14:18.838947+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911287+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:14:18.911287+0000","last_peered":"2024-09-05T01:14:18.911287+0000","last_clean":"2024-09-05T01:14:18.911287+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:14:18.911287+0000","last_undegraded":"2024-09-05T01:14:18.911287+0000","last_fullsized":"2024-09-05T01:14:18.911287+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834331+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:14:18.834331+0000","last_peered":"2024-09-05T01:14:18.834331+0000","last_clean":"2024-09-05T01:14:18.834331+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:14:18.834331+0000","last_undegraded":"2024-09-05T01:14:18.834331+0000","last_fullsized":"2024-09-05T01:14:18.834331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911176+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:14:18.911176+0000","last_peered":"2024-09-05T01:14:18.911176+0000","last_clean":"2024-09-05T01:14:18.911176+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:14:18.911176+0000","last_undegraded":"2024-09-05T01:14:18.911176+0000","last_fullsized":"2024-09-05T01:14:18.911176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910789+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:14:18.910789+0000","last_peered":"2024-09-05T01:14:18.910789+0000","last_clean":"2024-09-05T01:14:18.910789+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:14:18.910789+0000","last_undegraded":"2024-09-05T01:14:18.910789+0000","last_fullsized":"2024-09-05T01:14:18.910789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834612+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:14:18.834612+0000","last_peered":"2024-09-05T01:14:18.834612+0000","last_clean":"2024-09-05T01:14:18.834612+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:14:18.834612+0000","last_undegraded":"2024-09-05T01:14:18.834612+0000","last_fullsized":"2024-09-05T01:14:18.834612+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834389+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:14:18.834389+0000","last_peered":"2024-09-05T01:14:18.834389+0000","last_clean":"2024-09-05T01:14:18.834389+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:14:18.834389+0000","last_undegraded":"2024-09-05T01:14:18.834389+0000","last_fullsized":"2024-09-05T01:14:18.834389+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835027+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:14:18.835027+0000","last_peered":"2024-09-05T01:14:18.835027+0000","last_clean":"2024-09-05T01:14:18.835027+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:14:18.835027+0000","last_undegraded":"2024-09-05T01:14:18.835027+0000","last_fullsized":"2024-09-05T01:14:18.835027+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911083+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:14:18.911083+0000","last_peered":"2024-09-05T01:14:18.911083+0000","last_clean":"2024-09-05T01:14:18.911083+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:14:18.911083+0000","last_undegraded":"2024-09-05T01:14:18.911083+0000","last_fullsized":"2024-09-05T01:14:18.911083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911008+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:14:18.911008+0000","last_peered":"2024-09-05T01:14:18.911008+0000","last_clean":"2024-09-05T01:14:18.911008+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:14:18.911008+0000","last_undegraded":"2024-09-05T01:14:18.911008+0000","last_fullsized":"2024-09-05T01:14:18.911008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.837038+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:14:18.837038+0000","last_peered":"2024-09-05T01:14:18.837038+0000","last_clean":"2024-09-05T01:14:18.837038+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:14:18.837038+0000","last_undegraded":"2024-09-05T01:14:18.837038+0000","last_fullsized":"2024-09-05T01:14:18.837038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.910803+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:14:18.910803+0000","last_peered":"2024-09-05T01:14:18.910803+0000","last_clean":"2024-09-05T01:14:18.910803+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:14:18.910803+0000","last_undegraded":"2024-09-05T01:14:18.910803+0000","last_fullsized":"2024-09-05T01:14:18.910803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911444+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:14:18.911444+0000","last_peered":"2024-09-05T01:14:18.911444+0000","last_clean":"2024-09-05T01:14:18.911444+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:14:18.911444+0000","last_undegraded":"2024-09-05T01:14:18.911444+0000","last_fullsized":"2024-09-05T01:14:18.911444+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834732+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:14:18.834732+0000","last_peered":"2024-09-05T01:14:18.834732+0000","last_clean":"2024-09-05T01:14:18.834732+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:14:18.834732+0000","last_undegraded":"2024-09-05T01:14:18.834732+0000","last_fullsized":"2024-09-05T01:14:18.834732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.840355+0000","last_change":"2024-09-05T01:14:18.840355+0000","last_active":"2024-09-05T01:14:18.840355+0000","last_peered":"2024-09-05T01:14:18.840355+0000","last_clean":"2024-09-05T01:14:18.840355+0000","last_became_active":"2024-09-05T01:14:18.839887+0000","last_became_peered":"2024-09-05T01:14:18.839887+0000","last_unstale":"2024-09-05T01:14:18.840355+0000","last_undegraded":"2024-09-05T01:14:18.840355+0000","last_fullsized":"2024-09-05T01:14:18.840355+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:49.152575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911396+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:14:18.911396+0000","last_peered":"2024-09-05T01:14:18.911396+0000","last_clean":"2024-09-05T01:14:18.911396+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:14:18.911396+0000","last_undegraded":"2024-09-05T01:14:18.911396+0000","last_fullsized":"2024-09-05T01:14:18.911396+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.831789+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:14:18.831789+0000","last_peered":"2024-09-05T01:14:18.831789+0000","last_clean":"2024-09-05T01:14:18.831789+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:14:18.831789+0000","last_undegraded":"2024-09-05T01:14:18.831789+0000","last_fullsized":"2024-09-05T01:14:18.831789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835470+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:14:18.835470+0000","last_peered":"2024-09-05T01:14:18.835470+0000","last_clean":"2024-09-05T01:14:18.835470+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:14:18.835470+0000","last_undegraded":"2024-09-05T01:14:18.835470+0000","last_fullsized":"2024-09-05T01:14:18.835470+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911195+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:14:18.911195+0000","last_peered":"2024-09-05T01:14:18.911195+0000","last_clean":"2024-09-05T01:14:18.911195+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:14:18.911195+0000","last_undegraded":"2024-09-05T01:14:18.911195+0000","last_fullsized":"2024-09-05T01:14:18.911195+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833879+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:14:18.833879+0000","last_peered":"2024-09-05T01:14:18.833879+0000","last_clean":"2024-09-05T01:14:18.833879+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:14:18.833879+0000","last_undegraded":"2024-09-05T01:14:18.833879+0000","last_fullsized":"2024-09-05T01:14:18.833879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833934+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:14:18.833934+0000","last_peered":"2024-09-05T01:14:18.833934+0000","last_clean":"2024-09-05T01:14:18.833934+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:14:18.833934+0000","last_undegraded":"2024-09-05T01:14:18.833934+0000","last_fullsized":"2024-09-05T01:14:18.833934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.836841+0000","last_change":"2024-09-05T01:14:18.836841+0000","last_active":"2024-09-05T01:14:18.836841+0000","last_peered":"2024-09-05T01:14:18.836841+0000","last_clean":"2024-09-05T01:14:18.836841+0000","last_became_active":"2024-09-05T01:14:18.836464+0000","last_became_peered":"2024-09-05T01:14:18.836464+0000","last_unstale":"2024-09-05T01:14:18.836841+0000","last_undegraded":"2024-09-05T01:14:18.836841+0000","last_fullsized":"2024-09-05T01:14:18.836841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:21:02.858269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834220+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:14:18.834220+0000","last_peered":"2024-09-05T01:14:18.834220+0000","last_clean":"2024-09-05T01:14:18.834220+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:14:18.834220+0000","last_undegraded":"2024-09-05T01:14:18.834220+0000","last_fullsized":"2024-09-05T01:14:18.834220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.912623+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:14:18.912623+0000","last_peered":"2024-09-05T01:14:18.912623+0000","last_clean":"2024-09-05T01:14:18.912623+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:14:18.912623+0000","last_undegraded":"2024-09-05T01:14:18.912623+0000","last_fullsized":"2024-09-05T01:14:18.912623+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.833329+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:14:18.833329+0000","last_peered":"2024-09-05T01:14:18.833329+0000","last_clean":"2024-09-05T01:14:18.833329+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:14:18.833329+0000","last_undegraded":"2024-09-05T01:14:18.833329+0000","last_fullsized":"2024-09-05T01:14:18.833329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911174+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:14:18.911174+0000","last_peered":"2024-09-05T01:14:18.911174+0000","last_clean":"2024-09-05T01:14:18.911174+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:14:18.911174+0000","last_undegraded":"2024-09-05T01:14:18.911174+0000","last_fullsized":"2024-09-05T01:14:18.911174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911008+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:14:18.911008+0000","last_peered":"2024-09-05T01:14:18.911008+0000","last_clean":"2024-09-05T01:14:18.911008+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:14:18.911008+0000","last_undegraded":"2024-09-05T01:14:18.911008+0000","last_fullsized":"2024-09-05T01:14:18.911008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.911593+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:14:18.911593+0000","last_peered":"2024-09-05T01:14:18.911593+0000","last_clean":"2024-09-05T01:14:18.911593+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:14:18.911593+0000","last_undegraded":"2024-09-05T01:14:18.911593+0000","last_fullsized":"2024-09-05T01:14:18.911593+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.832918+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:14:18.832918+0000","last_peered":"2024-09-05T01:14:18.832918+0000","last_clean":"2024-09-05T01:14:18.832918+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:14:18.832918+0000","last_undegraded":"2024-09-05T01:14:18.832918+0000","last_fullsized":"2024-09-05T01:14:18.832918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.838715+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:14:18.838715+0000","last_peered":"2024-09-05T01:14:18.838715+0000","last_clean":"2024-09-05T01:14:18.838715+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:14:18.838715+0000","last_undegraded":"2024-09-05T01:14:18.838715+0000","last_fullsized":"2024-09-05T01:14:18.838715+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834110+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:14:18.834110+0000","last_peered":"2024-09-05T01:14:18.834110+0000","last_clean":"2024-09-05T01:14:18.834110+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:14:18.834110+0000","last_undegraded":"2024-09-05T01:14:18.834110+0000","last_fullsized":"2024-09-05T01:14:18.834110+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834212+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:14:18.834212+0000","last_peered":"2024-09-05T01:14:18.834212+0000","last_clean":"2024-09-05T01:14:18.834212+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:14:18.834212+0000","last_undegraded":"2024-09-05T01:14:18.834212+0000","last_fullsized":"2024-09-05T01:14:18.834212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.834721+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:14:18.834721+0000","last_peered":"2024-09-05T01:14:18.834721+0000","last_clean":"2024-09-05T01:14:18.834721+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:14:18.834721+0000","last_undegraded":"2024-09-05T01:14:18.834721+0000","last_fullsized":"2024-09-05T01:14:18.834721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-05T01:14:18.835458+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:14:18.835458+0000","last_peered":"2024-09-05T01:14:18.835458+0000","last_clean":"2024-09-05T01:14:18.835458+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:14:18.835458+0000","last_undegraded":"2024-09-05T01:14:18.835458+0000","last_fullsized":"2024-09-05T01:14:18.835458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":352,"num_read_kb":247,"num_write":167,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":495545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36996,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708156,"statfs":{"total":95995035648,"available":95957151744,"internally_reserved":0,"allocated":1687552,"data_stored":1144889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972394,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":499877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201328,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1015808,"data_stored":495701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430261,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1609728,"data_stored":1091612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593787,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":995328,"data_stored":490957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1617920,"data_stored":1087200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1007616,"data_stored":495607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215177,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1007616,"data_stored":491151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1085440,"data_stored":554686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1060864,"data_stored":552943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":492345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:14:46.012 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T01:14:46.012 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T01:14:46.012 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-05T01:14:46.012 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph health --format=json 2024-09-05T01:14:46.260 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- 172.21.15.129:0/4122374995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 msgr2=0x7f941c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 --2- 172.21.15.129:0/4122374995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c105920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f940c009900 tx=0x7f940c02a470 comp rx=0 tx=0).stop 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- 172.21.15.129:0/4122374995 shutdown_connections 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 --2- 172.21.15.129:0/4122374995 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 0x7f941c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 --2- 172.21.15.129:0/4122374995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c105920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 --2- 172.21.15.129:0/4122374995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- 172.21.15.129:0/4122374995 >> 172.21.15.129:0/4122374995 conn(0x7f941c0fc7a0 msgr2=0x7f941c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- 172.21.15.129:0/4122374995 shutdown_connections 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- 172.21.15.129:0/4122374995 wait complete. 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 Processor -- start 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 -- start start 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.714+0000 7f94239ae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c1a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 0x7f941c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f941c1183f0 con 0x7f941c101450 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f941c118270 con 0x7f941c100b30 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f941c118570 con 0x7f941c105f40 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47084/0 (socket says 172.21.15.129:47084) 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 -- 172.21.15.129:0/2611422442 learned_addr learned my addr 172.21.15.129:0/2611422442 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:46.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9421f4b640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 0x7f941c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f942174a640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c1a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 -- 172.21.15.129:0/2611422442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 msgr2=0x7f941c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 0x7f941c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 -- 172.21.15.129:0/2611422442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 msgr2=0x7f941c1a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c1a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 -- 172.21.15.129:0/2611422442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c19ef90 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f942174a640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f9420f49640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f940c006010 tx=0x7f940c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c03f030 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c19f220 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f940c003800 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c035070 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.715+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c1ab570 con 0x7f941c101450 2024-09-05T01:14:46.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.716+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f941c105920 con 0x7f941c101450 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.718+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f940c031af0 con 0x7f941c101450 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.718+0000 7f94127fc640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 0x7f94080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.718+0000 7f942174a640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 0x7f94080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.718+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f940c0efdd0 con 0x7f941c101450 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.719+0000 7f942174a640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 0x7f94080a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9418005fd0 tx=0x7f9418005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:46.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.722+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f940c0bc870 con 0x7f941c101450 2024-09-05T01:14:46.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.925+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f941c1ab740 con 0x7f941c101450 2024-09-05T01:14:46.926 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:46 smithi129 ceph-mon[30979]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:46.926 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:46 smithi129 ceph-mon[30979]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:46.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.925+0000 7f94127fc640 1 -- 172.21.15.129:0/2611422442 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f940c031de0 con 0x7f941c101450 2024-09-05T01:14:46.926 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:46.926 INFO:teuthology.orchestra.run.smithi129.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 msgr2=0x7f94080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 0x7f94080a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9418005fd0 tx=0x7f9418005ec0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 msgr2=0x7f941c1a4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f940c006010 tx=0x7f940c0042f0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 shutdown_connections 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94080a2c10 0x7f94080a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f941c105f40 0x7f941c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f941c101450 0x7f941c1a4be0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 --2- 172.21.15.129:0/2611422442 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f941c100b30 0x7f941c1a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 >> 172.21.15.129:0/2611422442 conn(0x7f941c0fc7a0 msgr2=0x7f941c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 shutdown_connections 2024-09-05T01:14:46.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:46.927+0000 7f94239ae640 1 -- 172.21.15.129:0/2611422442 wait complete. 2024-09-05T01:14:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:46 smithi183 ceph-mon[34589]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:46 smithi183 ceph-mon[34589]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:46 smithi194 ceph-mon[34941]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:14:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:46 smithi194 ceph-mon[34941]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:47.621 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-05T01:14:47.621 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:14:47.869 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2611422442' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T01:14:48.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:47 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2611422442' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T01:14:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:47 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2611422442' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/3711806318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 msgr2=0x7f1b4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/3711806318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b4410ea90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009900 tx=0x7f1b2c02a3f0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/3711806318 shutdown_connections 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/3711806318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b4410ea90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/3711806318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 0x7f1b441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/3711806318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/3711806318 >> 172.21.15.129:0/3711806318 conn(0x7f1b440fd8c0 msgr2=0x7f1b440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/3711806318 shutdown_connections 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.343+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/3711806318 wait complete. 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 Processor -- start 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 -- start start 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b440744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 0x7f1b44074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:48.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44111e20 con 0x7f1b44106f10 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44111ca0 con 0x7f1b44101de0 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b4ad8c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1b44111fa0 con 0x7f1b44102790 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47096/0 (socket says 172.21.15.129:47096) 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 -- 172.21.15.129:0/1226500923 learned_addr learned my addr 172.21.15.129:0/1226500923 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b48b28640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b440744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b43fff640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 0x7f1b44074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 -- 172.21.15.129:0/1226500923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 msgr2=0x7f1b44074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 0x7f1b44074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 -- 172.21.15.129:0/1226500923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 msgr2=0x7f1b440744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b440744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b49329640 1 -- 172.21.15.129:0/1226500923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b441a6680 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.344+0000 7f1b48b28640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b49329640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009a20 tx=0x7f1b2c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c03f030 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b441a6910 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b2c0044d0 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c035070 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.345+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b441a6e40 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.346+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b441021c0 con 0x7f1b44106f10 2024-09-05T01:14:48.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.347+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1b2c042030 con 0x7f1b44106f10 2024-09-05T01:14:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.348+0000 7f1b41ffb640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 0x7f1b300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.348+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f1b2c0f0180 con 0x7f1b44106f10 2024-09-05T01:14:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.348+0000 7f1b48b28640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 0x7f1b300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.348+0000 7f1b48b28640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 0x7f1b300a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00abe0 tx=0x7f1b3c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:48.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.351+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b2c0bcbc0 con 0x7f1b44106f10 2024-09-05T01:14:48.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.552+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b4402d7c0 con 0x7f1b44106f10 2024-09-05T01:14:48.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.553+0000 7f1b41ffb640 1 -- 172.21.15.129:0/1226500923 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f1b2c0bc4c0 con 0x7f1b44106f10 2024-09-05T01:14:48.554 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:48.554 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:14:48.554 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:14:48.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 msgr2=0x7f1b300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:48.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 0x7f1b300a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00abe0 tx=0x7f1b3c00a5c0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 msgr2=0x7f1b44074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009a20 tx=0x7f1b2c0042e0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 shutdown_connections 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.555+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1b300a2c10 0x7f1b300a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1b44106f10 0x7f1b44074f70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1b44102790 0x7f1b44074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 --2- 172.21.15.129:0/1226500923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1b44101de0 0x7f1b440744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 >> 172.21.15.129:0/1226500923 conn(0x7f1b440fd8c0 msgr2=0x7f1b440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 shutdown_connections 2024-09-05T01:14:48.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:48.556+0000 7f1b4ad8c640 1 -- 172.21.15.129:0/1226500923 wait complete. 2024-09-05T01:14:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:48 smithi183 ceph-mon[34589]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:49.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:48 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1226500923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:48 smithi129 ceph-mon[30979]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1226500923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:48 smithi194 ceph-mon[34941]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:48 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1226500923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:49.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:14:49.601 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- 172.21.15.129:0/2333499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4101420 msgr2=0x7efeb410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 --2- 172.21.15.129:0/2333499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4101420 0x7efeb410d950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efeac009cd0 tx=0x7efeac02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- 172.21.15.129:0/2333499984 shutdown_connections 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 --2- 172.21.15.129:0/2333499984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4101420 0x7efeb410d950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 --2- 172.21.15.129:0/2333499984 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efeb4100a80 0x7efeb4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 --2- 172.21.15.129:0/2333499984 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4106a80 0x7efeb4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- 172.21.15.129:0/2333499984 >> 172.21.15.129:0/2333499984 conn(0x7efeb40fc7a0 msgr2=0x7efeb40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- 172.21.15.129:0/2333499984 shutdown_connections 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- 172.21.15.129:0/2333499984 wait complete. 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 Processor -- start 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.060+0000 7efeba8ea640 1 -- start start 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4101420 0x7efeb4197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efeb4106a80 0x7efeb419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110dd0 con 0x7efeb4100a80 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110c50 con 0x7efeb4101420 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeba8ea640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4110f50 con 0x7efeb4106a80 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:50.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47110/0 (socket says 172.21.15.129:47110) 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 -- 172.21.15.129:0/1447297409 learned_addr learned my addr 172.21.15.129:0/1447297409 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb37fe640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4101420 0x7efeb4197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 -- 172.21.15.129:0/1447297409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efeb4106a80 msgr2=0x7efeb419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efeb4106a80 0x7efeb419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 -- 172.21.15.129:0/1447297409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4101420 msgr2=0x7efeb4197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4101420 0x7efeb4197930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 -- 172.21.15.129:0/1447297409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb419c3a0 con 0x7efeb4100a80 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.061+0000 7efeb3fff640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efea400d990 tx=0x7efea400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.062+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea401b070 con 0x7efeb4100a80 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.062+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efea400ed70 con 0x7efeb4100a80 2024-09-05T01:14:50.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.062+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb419c690 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.062+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea40146c0 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.062+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb406a790 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.063+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb4100f60 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.064+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efea401e070 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.064+0000 7efeb17fa640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 0x7efea00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.064+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7efea40d0ad0 con 0x7efeb4100a80 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.064+0000 7efeb37fe640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 0x7efea00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.065+0000 7efeb37fe640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 0x7efea00a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efeb40fee70 tx=0x7efe9c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:50.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.069+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efea409d680 con 0x7efeb4100a80 2024-09-05T01:14:50.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.272+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efeb406b210 con 0x7efeb4100a80 2024-09-05T01:14:50.273 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.273+0000 7efeb17fa640 1 -- 172.21.15.129:0/1447297409 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7efea409cf80 con 0x7efeb4100a80 2024-09-05T01:14:50.274 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:50.274 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:14:50.274 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 msgr2=0x7efea00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 0x7efea00a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efeb40fee70 tx=0x7efe9c009110 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 msgr2=0x7efeb41973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efea400d990 tx=0x7efea400dd90 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 shutdown_connections 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efea00a2c10 0x7efea00a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efeb4106a80 0x7efeb419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efeb4101420 0x7efeb4197930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 --2- 172.21.15.129:0/1447297409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efeb4100a80 0x7efeb41973f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 >> 172.21.15.129:0/1447297409 conn(0x7efeb40fc7a0 msgr2=0x7efeb40fd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:50.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 shutdown_connections 2024-09-05T01:14:50.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:50.275+0000 7efeba8ea640 1 -- 172.21.15.129:0/1447297409 wait complete. 2024-09-05T01:14:50.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:50 smithi129 ceph-mon[30979]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:50.987 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:50 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1447297409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:51.015 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:14:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:50 smithi183 ceph-mon[34589]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:51.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:50 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1447297409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:50 smithi194 ceph-mon[34941]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:50 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1447297409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:51.262 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 -- 172.21.15.129:0/1977721911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64106a80 msgr2=0x7f2a64106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 --2- 172.21.15.129:0/1977721911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64106a80 0x7f2a64106e60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009930 tx=0x7f2a5c02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 -- 172.21.15.129:0/1977721911 shutdown_connections 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 --2- 172.21.15.129:0/1977721911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64101420 0x7f2a6410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 --2- 172.21.15.129:0/1977721911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64100a80 0x7f2a64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 --2- 172.21.15.129:0/1977721911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64106a80 0x7f2a64106e60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 -- 172.21.15.129:0/1977721911 >> 172.21.15.129:0/1977721911 conn(0x7f2a640fc7a0 msgr2=0x7f2a640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 -- 172.21.15.129:0/1977721911 shutdown_connections 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.702+0000 7f2a689be640 1 -- 172.21.15.129:0/1977721911 wait complete. 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 Processor -- start 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 -- start start 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64100a80 0x7f2a641991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64106a80 0x7f2a641a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64110cf0 con 0x7f2a64101420 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64110b70 con 0x7f2a64106a80 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a689be640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64110e70 con 0x7f2a64100a80 2024-09-05T01:14:51.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a61d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a61d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47132/0 (socket says 172.21.15.129:47132) 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.703+0000 7f2a61d9b640 1 -- 172.21.15.129:0/3187068838 learned_addr learned my addr 172.21.15.129:0/3187068838 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a6259c640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64100a80 0x7f2a641991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a62d9d640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64106a80 0x7f2a641a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 -- 172.21.15.129:0/3187068838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64100a80 msgr2=0x7f2a641991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64100a80 0x7f2a641991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 -- 172.21.15.129:0/3187068838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64106a80 msgr2=0x7f2a641a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64106a80 0x7f2a641a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 -- 172.21.15.129:0/3187068838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a47a0 con 0x7f2a64101420 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a61d9b640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00d950 tx=0x7f2a4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c017030 con 0x7f2a64101420 2024-09-05T01:14:51.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a4970 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a4c004020 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0047a0 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.704+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a4eb0 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.705+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a64100ee0 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.706+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2a4c0041d0 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.707+0000 7f2a5b7fe640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 0x7f2a500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.707+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f2a4c0cef80 con 0x7f2a64101420 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.707+0000 7f2a6259c640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 0x7f2a500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.707+0000 7f2a6259c640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 0x7f2a500a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:51.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.711+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a4c09ba20 con 0x7f2a64101420 2024-09-05T01:14:51.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.881+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a64108ff0 con 0x7f2a64101420 2024-09-05T01:14:51.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.882+0000 7f2a5b7fe640 1 -- 172.21.15.129:0/3187068838 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24729 (secure 0 0 0) 0x7f2a4c09b320 con 0x7f2a64101420 2024-09-05T01:14:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:51.883 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:14:18.823732+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:14:51.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 msgr2=0x7f2a500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 0x7f2a500a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c000bc0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 msgr2=0x7f2a64199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00d950 tx=0x7f2a4c00de10 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 shutdown_connections 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2a500a2c10 0x7f2a500a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2a64106a80 0x7f2a641a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a64101420 0x7f2a64199720 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 --2- 172.21.15.129:0/3187068838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2a64100a80 0x7f2a641991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 >> 172.21.15.129:0/3187068838 conn(0x7f2a640fc7a0 msgr2=0x7f2a640fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 shutdown_connections 2024-09-05T01:14:51.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:51.885+0000 7f2a689be640 1 -- 172.21.15.129:0/3187068838 wait complete. 2024-09-05T01:14:52.599 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:14:52.845 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:53.090 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:52 smithi129 ceph-mon[30979]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:53.091 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:52 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3187068838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:52 smithi183 ceph-mon[34589]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:52 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3187068838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:52 smithi194 ceph-mon[34941]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:52 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3187068838' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:14:53.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 -- 172.21.15.129:0/3961311488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 msgr2=0x7f4b00100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:53.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 --2- 172.21.15.129:0/3961311488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00100ee0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4afc009930 tx=0x7f4afc02a420 comp rx=0 tx=0).stop 2024-09-05T01:14:53.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 -- 172.21.15.129:0/3961311488 shutdown_connections 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 --2- 172.21.15.129:0/3961311488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00101420 0x7f4b0010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 --2- 172.21.15.129:0/3961311488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00100ee0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 --2- 172.21.15.129:0/3961311488 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b00106a80 0x7f4b00106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 -- 172.21.15.129:0/3961311488 >> 172.21.15.129:0/3961311488 conn(0x7f4b000fc7a0 msgr2=0x7f4b000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 -- 172.21.15.129:0/3961311488 shutdown_connections 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.302+0000 7f4b08602640 1 -- 172.21.15.129:0/3961311488 wait complete. 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 Processor -- start 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 -- start start 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00101420 0x7f4b00198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b00106a80 0x7f4b0019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00110d70 con 0x7f4b00100a80 2024-09-05T01:14:53.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00110bf0 con 0x7f4b00101420 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b08602640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4b00110ef0 con 0x7f4b00106a80 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46198/0 (socket says 172.21.15.129:46198) 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 -- 172.21.15.129:0/2015006528 learned_addr learned my addr 172.21.15.129:0/2015006528 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 -- 172.21.15.129:0/2015006528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b00106a80 msgr2=0x7f4b0019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b00106a80 0x7f4b0019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 -- 172.21.15.129:0/2015006528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00101420 msgr2=0x7f4b00198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00101420 0x7f4b00198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 -- 172.21.15.129:0/2015006528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b0019d400 con 0x7f4b00100a80 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4b0639e640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4af000d770 tx=0x7f4af000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.303+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0017070 con 0x7f4b00100a80 2024-09-05T01:14:53.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.304+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af00054d0 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.304+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b0019d6f0 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.304+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0004cf0 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.304+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b001a4fd0 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.304+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b00100ee0 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.306+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4af0004e90 con 0x7f4b00100a80 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.306+0000 7f4af77fe640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 0x7f4aec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.306+0000 7f4b05b9d640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 0x7f4aec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:53.327 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.306+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f4af00cf4a0 con 0x7f4b00100a80 2024-09-05T01:14:53.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.307+0000 7f4b05b9d640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 0x7f4aec0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0096c0 tx=0x7f4afc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:53.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.310+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4af009bf40 con 0x7f4b00100a80 2024-09-05T01:14:53.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.511+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b001997b0 con 0x7f4b00100a80 2024-09-05T01:14:53.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.512+0000 7f4af77fe640 1 -- 172.21.15.129:0/2015006528 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7f4af009b840 con 0x7f4b00100a80 2024-09-05T01:14:53.513 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:14:53.513 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:14:53.514 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:14:53.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 msgr2=0x7f4aec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:53.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 0x7f4aec0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0096c0 tx=0x7f4afc031400 comp rx=0 tx=0).stop 2024-09-05T01:14:53.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 msgr2=0x7f4b00198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:53.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4af000d770 tx=0x7f4af000dc30 comp rx=0 tx=0).stop 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 shutdown_connections 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4aec0a2c10 0x7f4aec0a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4b00106a80 0x7f4b0019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4b00101420 0x7f4b00198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 --2- 172.21.15.129:0/2015006528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b00100a80 0x7f4b00198450 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 >> 172.21.15.129:0/2015006528 conn(0x7f4b000fc7a0 msgr2=0x7f4b0006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 shutdown_connections 2024-09-05T01:14:53.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:53.514+0000 7f4b08602640 1 -- 172.21.15.129:0/2015006528 wait complete. 2024-09-05T01:14:54.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:53 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2015006528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:54.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:53 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2015006528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:54.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:53 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2015006528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:14:54.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T01:13:06.500509+0000', 'modified': '2024-09-05T01:13:50.664418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24452, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3903364483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3903364483}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3903364483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/1641200590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 1641200590}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 1641200590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/979076613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 979076613}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 979076613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.194:6839/203556383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 203556383}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 203556383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.183:6837/685319691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 685319691}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 685319691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/3533858614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3533858614}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3533858614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24451, 24452, 24488]} 2024-09-05T01:14:54.317 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-05T01:14:54.317 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T01:14:54.563 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:54.975 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:54 smithi129 ceph-mon[30979]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:55.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 -- 172.21.15.129:0/1168809519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0100a80 msgr2=0x7f0da0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 --2- 172.21.15.129:0/1168809519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0100a80 0x7f0da0100ee0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009930 tx=0x7f0d9002a420 comp rx=0 tx=0).stop 2024-09-05T01:14:55.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 -- 172.21.15.129:0/1168809519 shutdown_connections 2024-09-05T01:14:55.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 --2- 172.21.15.129:0/1168809519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0101420 0x7f0da010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 --2- 172.21.15.129:0/1168809519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0100a80 0x7f0da0100ee0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 --2- 172.21.15.129:0/1168809519 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0106a80 0x7f0da0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.054+0000 7f0da731d640 1 -- 172.21.15.129:0/1168809519 >> 172.21.15.129:0/1168809519 conn(0x7f0da00fc7a0 msgr2=0x7f0da00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- 172.21.15.129:0/1168809519 shutdown_connections 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- 172.21.15.129:0/1168809519 wait complete. 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 Processor -- start 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- start start 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0100a80 0x7f0da00fced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0106a80 0x7f0da00fd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0110d30 con 0x7f0da0101420 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0110bb0 con 0x7f0da0106a80 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da731d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0da0110eb0 con 0x7f0da0100a80 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46218/0 (socket says 172.21.15.129:46218) 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.055+0000 7f0da48b8640 1 -- 172.21.15.129:0/4221757600 learned_addr learned my addr 172.21.15.129:0/4221757600 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da58ba640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0106a80 0x7f0da00fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da50b9640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0100a80 0x7f0da00fced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 -- 172.21.15.129:0/4221757600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0100a80 msgr2=0x7f0da00fced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0100a80 0x7f0da00fced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 -- 172.21.15.129:0/4221757600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0106a80 msgr2=0x7f0da00fd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0106a80 0x7f0da00fd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 -- 172.21.15.129:0/4221757600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da01a68c0 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da48b8640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0d90004770 tx=0x7f0d900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0d967fc640 1 -- 172.21.15.129:0/4221757600 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9003f070 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0d967fc640 1 -- 172.21.15.129:0/4221757600 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d9003b460 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da01a6b50 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0d967fc640 1 -- 172.21.15.129:0/4221757600 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90035070 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.056+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da01a7080 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.058+0000 7f0d967fc640 1 -- 172.21.15.129:0/4221757600 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0d9003b610 con 0x7f0da0101420 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 Processor -- start 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 -- start start 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba29d0 0x5616a1ba2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0d967fc640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 0x7f0d8c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1badbb0 0x5616a1baffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b09090 con 0x5616a1ba32f0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616a1ab0a70 con 0x5616a1badbb0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da96f6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616a1b05050 con 0x5616a1ba29d0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da50b9640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 0x7f0d8c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46228/0 (socket says 172.21.15.129:46228) 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da48b8640 1 -- 172.21.15.129:0/1344402285 learned_addr learned my addr 172.21.15.129:0/1344402285 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da58ba640 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1badbb0 0x5616a1baffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.059+0000 7f0da58ba640 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba29d0 0x5616a1ba2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0d967fc640 1 -- 172.21.15.129:0/4221757600 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d900ef8d0 con 0x7f0da0101420 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 -- 172.21.15.129:0/1344402285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba29d0 msgr2=0x5616a1ba2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba29d0 0x5616a1ba2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 -- 172.21.15.129:0/1344402285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1badbb0 msgr2=0x5616a1baffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1badbb0 0x5616a1baffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 -- 172.21.15.129:0/1344402285 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a1abd130 con 0x5616a1ba32f0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da48b8640 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d900b6710 tx=0x7f0d9002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c841e1fb9d1c14 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0da50b9640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 0x7f0d8c0a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005d70 tx=0x7f0d9c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0d8b7fe640 1 -- 172.21.15.129:0/1344402285 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90048070 con 0x5616a1ba32f0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0d8b7fe640 1 -- 172.21.15.129:0/1344402285 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d900bc370 con 0x5616a1ba32f0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.060+0000 7f0d8b7fe640 1 -- 172.21.15.129:0/1344402285 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900bbc70 con 0x5616a1ba32f0 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- 172.21.15.129:0/1344402285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 msgr2=0x5616a1bad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d900b6710 tx=0x7f0d9002a420 comp rx=0 tx=0).stop 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- 172.21.15.129:0/1344402285 shutdown_connections 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1badbb0 0x5616a1baffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bad670 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 --2- 172.21.15.129:0/1344402285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba29d0 0x5616a1ba2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- 172.21.15.129:0/1344402285 >> 172.21.15.129:0/1344402285 conn(0x5616a1b9ef60 msgr2=0x5616a1b9f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- 172.21.15.129:0/1344402285 shutdown_connections 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- 172.21.15.129:0/1344402285 wait complete. 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 -- start start 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.061+0000 7f0da96f6740 1 Processor -- start 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bad370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1baa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bb0540 con 0x5616a1badbb0 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616a1ac3f20 con 0x5616a1ba29d0 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bb06c0 con 0x5616a1ba32f0 2024-09-05T01:14:55.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 10 client.0 _ll_register_callbacks cb 0x5616a1a8d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46240/0 (socket says 172.21.15.129:46240) 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 -- 172.21.15.129:0/442281232 learned_addr learned my addr 172.21.15.129:0/442281232 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da48b8640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1baa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da50b9640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 -- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 msgr2=0x5616a1baa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1baa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 -- 172.21.15.129:0/442281232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 msgr2=0x5616a1bad370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bad370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 -- 172.21.15.129:0/442281232 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a187a2b0 con 0x5616a1badbb0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da48b8640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1baa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da58ba640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009270 tx=0x7f0d98004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302233c66a623f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98004830 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d98012b60 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.062+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d900bc370 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d98025050 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 10 client.15072 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616a1bbbd90 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d900bc370 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d9800f410 con 0x5616a1badbb0 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 10 client.15072 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 20 client.15072 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1bbf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.076 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 4 client.15072 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616a1b16600 con 0x5616a1bbe3d0 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.063+0000 7f0da50b9640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1bbf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.064+0000 7f0d88ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.064+0000 7f0da50b9640 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1bbf840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33793dac6b9bd176 server_cookie=8dafa02a53bc68aa in_seq=0 out_seq=0 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0d88ff9640 1 -- 172.21.15.129:0/442281232 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616a1b16600 con 0x5616a1bbe3d0 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0d88ff9640 10 client.15072 handle_command_reply: tid=0 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 client.15072 shutdown 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 msgr2=0x5616a1baac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009270 tx=0x7f0d98004700 comp rx=0 tx=0).stop 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 shutdown_connections 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1bbf840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1baac00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0d88ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:14:55.077 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1baa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 --2- 172.21.15.129:0/442281232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bad370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 >> 172.21.15.129:0/442281232 conn(0x5616a1b9ef60 msgr2=0x5616a1ba4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 shutdown_connections 2024-09-05T01:14:55.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 1 -- 172.21.15.129:0/442281232 wait complete. 2024-09-05T01:14:55.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.065+0000 7f0da96f6740 20 client.15072 trim_cache size 0 max 16384 2024-09-05T01:14:55.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 Processor -- start 2024-09-05T01:14:55.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 -- start start 2024-09-05T01:14:55.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bbf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1bb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.084 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbdb80 con 0x5616a1badbb0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbe250 con 0x5616a1ba29d0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da96f6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbe920 con 0x5616a1ba32f0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46244/0 (socket says 172.21.15.129:46244) 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 -- 172.21.15.129:0/2111635828 learned_addr learned my addr 172.21.15.129:0/2111635828 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da50b9640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bbf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da48b8640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1bb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 -- 172.21.15.129:0/2111635828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 msgr2=0x5616a1bb9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1bb9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 -- 172.21.15.129:0/2111635828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 msgr2=0x5616a1bbf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.085 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bbf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 -- 172.21.15.129:0/2111635828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616a1b9ed90 con 0x5616a1badbb0 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da48b8640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1bb9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.083+0000 7f0da58ba640 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d98023040 tx=0x7f0d98009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e07660ad75d7d69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0d8affd640 1 -- 172.21.15.129:0/2111635828 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800fea0 con 0x5616a1badbb0 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0d8affd640 1 -- 172.21.15.129:0/2111635828 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d98003ce0 con 0x5616a1badbb0 2024-09-05T01:14:55.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0d8affd640 1 -- 172.21.15.129:0/2111635828 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98012dc0 con 0x5616a1badbb0 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 -- 172.21.15.129:0/2111635828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 msgr2=0x5616a1b99ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d98023040 tx=0x7f0d98009520 comp rx=0 tx=0).stop 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 -- 172.21.15.129:0/2111635828 shutdown_connections 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1badbb0 0x5616a1b99ce0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1ba32f0 0x5616a1bb9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 --2- 172.21.15.129:0/2111635828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bbf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 -- 172.21.15.129:0/2111635828 >> 172.21.15.129:0/2111635828 conn(0x5616a1b9ef60 msgr2=0x5616a1baa680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 -- 172.21.15.129:0/2111635828 shutdown_connections 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.084+0000 7f0da96f6740 1 -- 172.21.15.129:0/2111635828 wait complete. 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 -- start start 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 Processor -- start 2024-09-05T01:14:55.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bc3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1badbb0 0x5616a1c20aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbf6c0 con 0x5616a1ba32f0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbeff0 con 0x5616a1ba29d0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5616a1bbfd90 con 0x5616a1badbb0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da96f6740 10 client.0 _ll_register_callbacks cb 0x7f0d98009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46256/0 (socket says 172.21.15.129:46256) 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 -- 172.21.15.129:0/3952306073 learned_addr learned my addr 172.21.15.129:0/3952306073 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da50b9640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bc3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da58ba640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1badbb0 0x5616a1c20aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 -- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1badbb0 msgr2=0x5616a1c20aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1badbb0 0x5616a1c20aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 -- 172.21.15.129:0/3952306073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 msgr2=0x5616a1bc3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bc3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.085+0000 7f0da48b8640 1 -- 172.21.15.129:0/3952306073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616a1c21110 con 0x5616a1ba32f0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da48b8640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ecfa0 tx=0x7f0d900095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829365f32bc922b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d90048040 con 0x5616a1ba32f0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d900ed170 con 0x5616a1ba32f0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d98003ce0 con 0x5616a1ba32f0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d900bc370 con 0x5616a1ba32f0 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 10 client.15078 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:55.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5616a1c283c0 con 0x5616a1ba32f0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d98003ce0 con 0x5616a1ba32f0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0d9004b060 con 0x5616a1ba32f0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 10 client.15078 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 20 client.15078 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1c2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 4 client.15078 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.086+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "status"}) -- 0x5616a1b10b10 con 0x5616a1bbe3d0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0da50b9640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1c2bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0d88ff9640 10 client.15078 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0da50b9640 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1c2bb50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c9435b0ba528e23 server_cookie=56eb78c39edf9b1 in_seq=0 out_seq=0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0d88ff9640 1 -- 172.21.15.129:0/3952306073 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5616a1b10b10 con 0x5616a1bbe3d0 2024-09-05T01:14:55.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0d88ff9640 10 client.15078 handle_command_reply: tid=0 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0da96f6740 1 client.15078 shutdown 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 msgr2=0x5616a1bc37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.087+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0d900ecfa0 tx=0x7f0d900095e0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 shutdown_connections 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x5616a1bbe3d0 0x5616a1c2bb50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5616a1badbb0 0x5616a1c20aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5616a1ba32f0 0x5616a1bc37c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0d88ff9640 0 client.15078 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 --2- 172.21.15.129:0/3952306073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5616a1ba29d0 0x5616a1bc3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 >> 172.21.15.129:0/3952306073 conn(0x5616a1b9ef60 msgr2=0x5616a1ba8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 shutdown_connections 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 1 -- 172.21.15.129:0/3952306073 wait complete. 2024-09-05T01:14:55.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da96f6740 20 client.15078 trim_cache size 0 max 16384 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 2, 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24451, 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 67.242372763000006, 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-09-05T01:14:55.095 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:14:55.096 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 127.447398333 2024-09-05T01:14:55.096 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:14:55.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 msgr2=0x7f0d8c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 0x7f0d8c0a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c005d70 tx=0x7f0d9c005d00 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 msgr2=0x7f0da00fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0d90004770 tx=0x7f0d900042f0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 shutdown_connections 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d8c0a2c90 0x7f0d8c0a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da0106a80 0x7f0da00fd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0da0101420 0x7f0da00fd410 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 --2- 172.21.15.129:0/4221757600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0da0100a80 0x7f0da00fced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 >> 172.21.15.129:0/4221757600 conn(0x7f0da00fc7a0 msgr2=0x7f0da006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.088+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 shutdown_connections 2024-09-05T01:14:55.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:55.089+0000 7f0da731d640 1 -- 172.21.15.129:0/4221757600 wait complete. 2024-09-05T01:14:55.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:54 smithi183 ceph-mon[34589]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:54 smithi194 ceph-mon[34941]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:14:55.877 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T01:14:56.124 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:56.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 -- 172.21.15.129:0/1104390274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0100a80 msgr2=0x7f58e0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1104390274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0100a80 0x7f58e0100ee0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009930 tx=0x7f58c802a420 comp rx=0 tx=0).stop 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 -- 172.21.15.129:0/1104390274 shutdown_connections 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1104390274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1104390274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0100a80 0x7f58e0100ee0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1104390274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0106a80 0x7f58e0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 -- 172.21.15.129:0/1104390274 >> 172.21.15.129:0/1104390274 conn(0x7f58e00fc7a0 msgr2=0x7f58e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 -- 172.21.15.129:0/1104390274 shutdown_connections 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.614+0000 7f58e48bd640 1 -- 172.21.15.129:0/1104390274 wait complete. 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 Processor -- start 2024-09-05T01:14:56.636 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 -- start start 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0100a80 0x7f58e01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0110cf0 con 0x7f58e0106a80 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0110b70 con 0x7f58e0100a80 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58e48bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f58e0110e70 con 0x7f58e0101420 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58ded9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58ded9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46280/0 (socket says 172.21.15.129:46280) 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58ded9d640 1 -- 172.21.15.129:0/1032405329 learned_addr learned my addr 172.21.15.129:0/1032405329 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.615+0000 7f58de59c640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0100a80 0x7f58e01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58d7fff640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 -- 172.21.15.129:0/1032405329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 msgr2=0x7f58e0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 -- 172.21.15.129:0/1032405329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0100a80 msgr2=0x7f58e01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0100a80 0x7f58e01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 -- 172.21.15.129:0/1032405329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e019d380 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58d7fff640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58ded9d640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58d800cad0 tx=0x7f58d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58d77fe640 1 -- 172.21.15.129:0/1032405329 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d800ae20 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58d77fe640 1 -- 172.21.15.129:0/1032405329 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d8005020 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e019d670 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58d77fe640 1 -- 172.21.15.129:0/1032405329 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d800f6b0 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.616+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e01a4f50 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.618+0000 7f58d77fe640 1 -- 172.21.15.129:0/1032405329 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f58d80051d0 con 0x7f58e0106a80 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.618+0000 7f58e6c96740 1 Processor -- start 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 -- start start 2024-09-05T01:14:56.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d75777bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 0x55d75778aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d7576e2090 con 0x55d75778afc0 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d757689a70 con 0x55d75777b8f0 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58e6c96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d7576de050 con 0x55d75777c210 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d77fe640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46284/0 (socket says 172.21.15.129:46284) 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d7fff640 1 -- 172.21.15.129:0/2679656061 learned_addr learned my addr 172.21.15.129:0/2679656061 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d75777bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58de59c640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 0x55d75778aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58ded9d640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d77fe640 1 -- 172.21.15.129:0/1032405329 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f58d80cf5d0 con 0x7f58e0106a80 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.619+0000 7f58d7fff640 1 -- 172.21.15.129:0/2679656061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 msgr2=0x55d75778aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 0x55d75778aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d7fff640 1 -- 172.21.15.129:0/2679656061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 msgr2=0x55d75777bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d75777bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d7fff640 1 -- 172.21.15.129:0/2679656061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d757696130 con 0x55d75778afc0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58de59c640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 0x55d75778aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58ded9d640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58d809b900 tx=0x7f58d80665f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58c80057c0 tx=0x7f58c80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4385bb12f364185a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d4ff9640 1 -- 172.21.15.129:0/2679656061 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8035040 con 0x55d75778afc0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d4ff9640 1 -- 172.21.15.129:0/2679656061 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58c80035f0 con 0x55d75778afc0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.620+0000 7f58d4ff9640 1 -- 172.21.15.129:0/2679656061 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c80044e0 con 0x55d75778afc0 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- 172.21.15.129:0/2679656061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 msgr2=0x55d75778d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.640 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58c80057c0 tx=0x7f58c80058d0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- 172.21.15.129:0/2679656061 shutdown_connections 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d75778d3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777c210 0x55d75778aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2679656061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d75777bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- 172.21.15.129:0/2679656061 >> 172.21.15.129:0/2679656061 conn(0x55d757777f60 msgr2=0x55d757778340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- 172.21.15.129:0/2679656061 shutdown_connections 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- 172.21.15.129:0/2679656061 wait complete. 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 -- start start 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.621+0000 7f58e6c96740 1 Processor -- start 2024-09-05T01:14:56.641 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777b8f0 0x55d75778a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777c210 0x55d757787c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d75778dad0 con 0x55d75778afc0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d75769cf20 con 0x55d75777c210 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d75778dc50 con 0x55d75777b8f0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 10 client.0 _ll_register_callbacks cb 0x55d757666030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46290/0 (socket says 172.21.15.129:46290) 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 -- 172.21.15.129:0/3887260406 learned_addr learned my addr 172.21.15.129:0/3887260406 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58de59c640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777c210 0x55d757787c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777b8f0 0x55d75778a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 -- 172.21.15.129:0/3887260406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777b8f0 msgr2=0x55d75778a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.642 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777b8f0 0x55d75778a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 -- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777c210 msgr2=0x55d757787c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777c210 0x55d757787c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 -- 172.21.15.129:0/3887260406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7574532b0 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58d7fff640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58c8001eb0 tx=0x7f58c8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df10e8fba21fc225 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c8055060 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f58c8003f70 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c80035f0 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58c8035070 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 10 client.15090 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.622+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7577994a0 con 0x55d75778afc0 2024-09-05T01:14:56.643 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58c80035f0 con 0x55d75778afc0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f58c803a040 con 0x55d75778afc0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 20 client.15090 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d75779a7f0 0x55d75779c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 4 client.15090 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7576ef600 con 0x55d75779a7f0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.623+0000 7f58de59c640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d75779a7f0 0x55d75779c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58de59c640 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d75779a7f0 0x55d75779c930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe5f03972651675a server_cookie=fa4b70d4f49241a in_seq=0 out_seq=0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58c5ffb640 10 client.15090 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58c5ffb640 1 -- 172.21.15.129:0/3887260406 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7576ef600 con 0x55d75779a7f0 2024-09-05T01:14:56.644 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58c5ffb640 10 client.15090 handle_command_reply: tid=0 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58e6c96740 1 client.15090 shutdown 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 msgr2=0x55d757788180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58c8001eb0 tx=0x7f58c8003e60 comp rx=0 tx=0).stop 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 shutdown_connections 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.624+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d75779a7f0 0x55d75779c930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75778afc0 0x55d757788180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777c210 0x55d757787c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58c5ffb640 0 client.15090 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 --2- 172.21.15.129:0/3887260406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75777b8f0 0x55d75778a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 >> 172.21.15.129:0/3887260406 conn(0x55d757777f60 msgr2=0x55d75777dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 shutdown_connections 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 1 -- 172.21.15.129:0/3887260406 wait complete. 2024-09-05T01:14:56.645 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.625+0000 7f58e6c96740 20 client.15090 trim_cache size 0 max 16384 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.642+0000 7f58e6c96740 1 Processor -- start 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.642+0000 7f58e6c96740 1 -- start start 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757787500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d757772ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d75779ac70 con 0x55d75777c210 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d75779b340 con 0x55d75777b8f0 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58e6c96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d75779ba10 con 0x55d75778afc0 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46304/0 (socket says 172.21.15.129:46304) 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 -- 172.21.15.129:0/176204292 learned_addr learned my addr 172.21.15.129:0/176204292 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58d7fff640 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757787500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 -- 172.21.15.129:0/176204292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 msgr2=0x55d757772ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d757772ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 -- 172.21.15.129:0/176204292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 msgr2=0x55d757787500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757787500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 -- 172.21.15.129:0/176204292 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d75778dc40 con 0x55d75777c210 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58de59c640 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58d000bd40 tx=0x7f58d00139a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f861b07bc9b577d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58c7fff640 1 -- 172.21.15.129:0/176204292 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0005020 con 0x55d75777c210 2024-09-05T01:14:56.651 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58c7fff640 1 -- 172.21.15.129:0/176204292 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d00047a0 con 0x55d75777c210 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.643+0000 7f58c7fff640 1 -- 172.21.15.129:0/176204292 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0010750 con 0x55d75777c210 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- 172.21.15.129:0/176204292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 msgr2=0x55d757783240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f58d000bd40 tx=0x7f58d00139a0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- 172.21.15.129:0/176204292 shutdown_connections 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d757772ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757783240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 --2- 172.21.15.129:0/176204292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757787500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- 172.21.15.129:0/176204292 >> 172.21.15.129:0/176204292 conn(0x55d757777f60 msgr2=0x55d757778340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- 172.21.15.129:0/176204292 shutdown_connections 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- 172.21.15.129:0/176204292 wait complete. 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.644+0000 7f58e6c96740 1 -- start start 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 Processor -- start 2024-09-05T01:14:56.652 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757784610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d7577a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d75779c7b0 con 0x55d75777c210 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d75779c0e0 con 0x55d75777b8f0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d75779ce80 con 0x55d75778afc0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58e6c96740 10 client.0 _ll_register_callbacks cb 0x7f58c8032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46308/0 (socket says 172.21.15.129:46308) 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 -- 172.21.15.129:0/2588085026 learned_addr learned my addr 172.21.15.129:0/2588085026 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757784610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d7577a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.653 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 -- 172.21.15.129:0/2588085026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 msgr2=0x55d7577a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d7577a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 -- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 msgr2=0x55d757784610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757784610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 -- 172.21.15.129:0/2588085026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7577a4230 con 0x55d75777c210 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757784610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58de59c640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58d0013390 tx=0x7f58d00133c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb66c763c529ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.645+0000 7f58c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d002b070 con 0x55d75777c210 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f58d00106f0 con 0x55d75777c210 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d00047a0 con 0x55d75777c210 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58d0037050 con 0x55d75777c210 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:56.654 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d7577a5510 con 0x55d75777c210 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58d00047a0 con 0x55d75777c210 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f58d0005410 con 0x55d75777c210 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 10 client.15096 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 20 client.15096 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d757809f50 0x55d757808c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 4 client.15096 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.646+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "status"}) -- 0x55d7576e9b10 con 0x55d757809f50 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d757809f50 0x55d757808c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58c5ffb640 10 client.15096 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58d7fff640 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d757809f50 0x55d757808c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a25f4005ad515b5 server_cookie=fa838bcf3a956b95 in_seq=0 out_seq=0 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58c5ffb640 1 -- 172.21.15.129:0/2588085026 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d7576e9b10 con 0x55d757809f50 2024-09-05T01:14:56.655 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58c5ffb640 10 client.15096 handle_command_reply: tid=0 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58e6c96740 1 client.15096 shutdown 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 msgr2=0x55d757784b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.647+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58d0013390 tx=0x7f58d00133c0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 shutdown_connections 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x55d757809f50 0x55d757808c60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d75778afc0 0x55d7577a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d75777c210 0x55d757784b50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 --2- 172.21.15.129:0/2588085026 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d75777b8f0 0x55d757784610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 >> 172.21.15.129:0/2588085026 conn(0x55d757777f60 msgr2=0x55d757783240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58c5ffb640 0 client.15096 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 shutdown_connections 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 1 -- 172.21.15.129:0/2588085026 wait complete. 2024-09-05T01:14:56.656 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e6c96740 20 client.15096 trim_cache size 0 max 16384 2024-09-05T01:14:56.660 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:14:56.660 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:14:56.660 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 1, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24452, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 69.813190180999996, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 131.635024367 2024-09-05T01:14:56.661 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 msgr2=0x7f58cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f58d809b900 tx=0x7f58d80665f0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 msgr2=0x7f58e019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f58d800cad0 tx=0x7f58d800cf90 comp rx=0 tx=0).stop 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 shutdown_connections 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f58cc0a2c10 0x7f58cc0a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f58e0106a80 0x7f58e019cca0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f58e0101420 0x7f58e0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 --2- 172.21.15.129:0/1032405329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f58e0100a80 0x7f58e01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.648+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 >> 172.21.15.129:0/1032405329 conn(0x7f58e00fc7a0 msgr2=0x7f58e006ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.649+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 shutdown_connections 2024-09-05T01:14:56.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:56.649+0000 7f58e48bd640 1 -- 172.21.15.129:0/1032405329 wait complete. 2024-09-05T01:14:57.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:56 smithi183 ceph-mon[34589]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:56 smithi129 ceph-mon[30979]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:56 smithi194 ceph-mon[34941]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:14:57.427 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T01:14:57.671 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 -- 172.21.15.129:0/1680725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c106f20 msgr2=0x7f489c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 --2- 172.21.15.129:0/1680725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c106f20 0x7f489c10eaa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4894009900 tx=0x7f489402a410 comp rx=0 tx=0).stop 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 -- 172.21.15.129:0/1680725669 shutdown_connections 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 --2- 172.21.15.129:0/1680725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c106f20 0x7f489c10eaa0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 --2- 172.21.15.129:0/1680725669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c1027a0 0x7f489c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 --2- 172.21.15.129:0/1680725669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c101df0 0x7f489c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 -- 172.21.15.129:0/1680725669 >> 172.21.15.129:0/1680725669 conn(0x7f489c0fd8f0 msgr2=0x7f489c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 -- 172.21.15.129:0/1680725669 shutdown_connections 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 -- 172.21.15.129:0/1680725669 wait complete. 2024-09-05T01:14:58.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.124+0000 7f48a10c3640 1 Processor -- start 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 -- start start 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c1027a0 0x7f489c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c106f20 0x7f489c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111e30 con 0x7f489c101df0 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111cb0 con 0x7f489c1027a0 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f48a10c3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111fb0 con 0x7f489c106f20 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46314/0 (socket says 172.21.15.129:46314) 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 -- 172.21.15.129:0/2817773300 learned_addr learned my addr 172.21.15.129:0/2817773300 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489a59c640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c1027a0 0x7f489c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 -- 172.21.15.129:0/2817773300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c106f20 msgr2=0x7f489c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c106f20 0x7f489c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 -- 172.21.15.129:0/2817773300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c1027a0 msgr2=0x7f489c198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c1027a0 0x7f489c198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 -- 172.21.15.129:0/2817773300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c19d2b0 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eab0 tx=0x7f488c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.125+0000 7f4883fff640 1 -- 172.21.15.129:0/2817773300 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c00ce40 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.126+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c19d5a0 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.126+0000 7f4883fff640 1 -- 172.21.15.129:0/2817773300 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f488c011920 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.126+0000 7f4883fff640 1 -- 172.21.15.129:0/2817773300 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c010a00 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.126+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c1a4e80 con 0x7f489c101df0 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 Processor -- start 2024-09-05T01:14:58.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f4883fff640 1 -- 172.21.15.129:0/2817773300 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f488c011ad0 con 0x7f489c101df0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 -- start start 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d4e9d0 0x558076d4edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d59bb0 0x558076d5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558076cb5090 con 0x558076d4f2f0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558076c5ca70 con 0x558076d59bb0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.128+0000 7f48a349c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558076cb1050 con 0x558076d4e9d0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46326/0 (socket says 172.21.15.129:46326) 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 -- 172.21.15.129:0/3243291497 learned_addr learned my addr 172.21.15.129:0/3243291497 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489b59e640 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d4e9d0 0x558076d4edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489a59c640 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d59bb0 0x558076d5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 -- 172.21.15.129:0/3243291497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d4e9d0 msgr2=0x558076d4edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d4e9d0 0x558076d4edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 -- 172.21.15.129:0/3243291497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d59bb0 msgr2=0x558076d5bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d59bb0 0x558076d5bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f4883fff640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 0x7f48840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 -- 172.21.15.129:0/3243291497 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558076c69130 con 0x558076d4f2f0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489b59e640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 0x7f48840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f489ad9d640 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f488c0cd190 tx=0x7f488c098300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa244c25c1a23f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f4883fff640 1 -- 172.21.15.129:0/2817773300 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f488c0cfc00 con 0x7f489c101df0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f48817fa640 1 -- 172.21.15.129:0/3243291497 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c09bd70 con 0x558076d4f2f0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f48817fa640 1 -- 172.21.15.129:0/3243291497 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f488c0a36d0 con 0x558076d4f2f0 2024-09-05T01:14:58.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.129+0000 7f48817fa640 1 -- 172.21.15.129:0/3243291497 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c09b900 con 0x558076d4f2f0 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- 172.21.15.129:0/3243291497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 msgr2=0x558076d59670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f489b59e640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 0x7f48840a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48940096c0 tx=0x7f4894031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f488c0cd190 tx=0x7f488c098300 comp rx=0 tx=0).stop 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- 172.21.15.129:0/3243291497 shutdown_connections 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d59bb0 0x558076d5bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d59670 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 --2- 172.21.15.129:0/3243291497 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d4e9d0 0x558076d4edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- 172.21.15.129:0/3243291497 >> 172.21.15.129:0/3243291497 conn(0x558076d4af60 msgr2=0x558076d4b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- 172.21.15.129:0/3243291497 shutdown_connections 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- 172.21.15.129:0/3243291497 wait complete. 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.130+0000 7f48a349c740 1 -- start start 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 Processor -- start 2024-09-05T01:14:58.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558076d5c780 con 0x558076d4f2f0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558076c6ff20 con 0x558076d4e9d0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558076d5c900 con 0x558076d59bb0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f48a349c740 10 client.0 _ll_register_callbacks cb 0x558076c39030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46330/0 (socket says 172.21.15.129:46330) 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 -- 172.21.15.129:0/2735592207 learned_addr learned my addr 172.21.15.129:0/2735592207 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489b59e640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d56c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 -- 172.21.15.129:0/2735592207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 msgr2=0x558076d56c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d56c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 -- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 msgr2=0x558076d593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 -- 172.21.15.129:0/2735592207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558076a262b0 con 0x558076d4f2f0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489b59e640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f489a59c640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4890011740 tx=0x7f4890011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc3005fe390a286 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.131+0000 7f487eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489001e060 con 0x558076d4f2f0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f489000fdb0 con 0x558076d4f2f0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558076cc2600 con 0x558076d4f2f0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f48900193f0 con 0x558076d4f2f0 2024-09-05T01:14:58.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 10 client.15108 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558076d68090 con 0x558076d4f2f0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558076cc2600 con 0x558076d4f2f0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4890018030 con 0x558076d4f2f0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 20 client.15108 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076d6a6e0 0x558076d6bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 4 client.15108 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.132+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558076cbcb10 con 0x558076d6a6e0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.133+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076d6a6e0 0x558076d6bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.133+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076d6a6e0 0x558076d6bb50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8495b83d70910dd2 server_cookie=fd825cba7d261395 in_seq=0 out_seq=0 2024-09-05T01:14:58.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.133+0000 7f487eff5640 10 client.15108 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f487eff5640 1 -- 172.21.15.129:0/2735592207 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558076cbcb10 con 0x558076d6a6e0 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f487eff5640 10 client.15108 handle_command_reply: tid=0 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 client.15108 shutdown 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 msgr2=0x558076d56720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4890011740 tx=0x7f4890011c00 comp rx=0 tx=0).stop 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 shutdown_connections 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076d6a6e0 0x558076d6bb50 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d56c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d56720 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f487eff5640 0 client.15108 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 --2- 172.21.15.129:0/2735592207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 >> 172.21.15.129:0/2735592207 conn(0x558076d4af60 msgr2=0x558076d50c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 shutdown_connections 2024-09-05T01:14:58.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 1 -- 172.21.15.129:0/2735592207 wait complete. 2024-09-05T01:14:58.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.134+0000 7f48a349c740 20 client.15108 trim_cache size 0 max 16384 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 Processor -- start 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 -- start start 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d6bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d51c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558076d69e90 con 0x558076d4f2f0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558076d6a560 con 0x558076d4e9d0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f48a349c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558076d6ac30 con 0x558076d59bb0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46332/0 (socket says 172.21.15.129:46332) 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 -- 172.21.15.129:0/2319880000 learned_addr learned my addr 172.21.15.129:0/2319880000 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489b59e640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d6bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 -- 172.21.15.129:0/2319880000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 msgr2=0x558076d51c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d51c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 -- 172.21.15.129:0/2319880000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 msgr2=0x558076d6bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d6bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 -- 172.21.15.129:0/2319880000 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558076d4ad90 con 0x558076d4f2f0 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d6bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.152+0000 7f489a59c640 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f48900056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7f6e093c2611c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f4880ff9640 1 -- 172.21.15.129:0/2319880000 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489002e030 con 0x558076d4f2f0 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f4880ff9640 1 -- 172.21.15.129:0/2319880000 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4890011e50 con 0x558076d4f2f0 2024-09-05T01:14:58.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f4880ff9640 1 -- 172.21.15.129:0/2319880000 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4890004280 con 0x558076d4f2f0 2024-09-05T01:14:58.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- 172.21.15.129:0/2319880000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 msgr2=0x558076d4b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4890000c00 tx=0x7f48900056f0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- 172.21.15.129:0/2319880000 shutdown_connections 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d51c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4f2f0 0x558076d4b340 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 --2- 172.21.15.129:0/2319880000 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4e9d0 0x558076d6bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- 172.21.15.129:0/2319880000 >> 172.21.15.129:0/2319880000 conn(0x558076d4af60 msgr2=0x558076d51560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- 172.21.15.129:0/2319880000 shutdown_connections 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- 172.21.15.129:0/2319880000 wait complete. 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.153+0000 7f48a349c740 1 -- start start 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 Processor -- start 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4f2f0 0x558076d536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558076d6b9d0 con 0x558076d4e9d0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558076d6b300 con 0x558076d4f2f0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558076d6c0a0 con 0x558076d59bb0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f48a349c740 10 client.0 _ll_register_callbacks cb 0x7f489000cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46344/0 (socket says 172.21.15.129:46344) 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 -- 172.21.15.129:0/2835068722 learned_addr learned my addr 172.21.15.129:0/2835068722 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489a59c640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4f2f0 0x558076d536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 -- 172.21.15.129:0/2835068722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 msgr2=0x558076d72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 -- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4f2f0 msgr2=0x558076d536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4f2f0 0x558076d536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 -- 172.21.15.129:0/2835068722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558076d73490 con 0x558076d4e9d0 2024-09-05T01:14:58.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f489ad9d640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f488c0cd190 tx=0x7f488c0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6d26b2265dfbbe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f487eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.154+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c09bae0 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f488c09a2a0 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4890011e50 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f488c09b900 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 10 client.15114 fetch_fsmap learned FSMap version 25 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558076d746e0 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4890011e50 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f488c0d5070 con 0x558076d4e9d0 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 10 client.15114 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 20 client.15114 populate_metadata read hostname 'smithi129' 2024-09-05T01:14:58.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076dd9000 0x558076dd7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 4 client.15114 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.155+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "status"}) -- 0x558076c390e0 con 0x558076dd9000 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f489a59c640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076dd9000 0x558076dd7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f489a59c640 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076dd9000 0x558076dd7d10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd23eb4c8df2ecd server_cookie=869f2b02dd070bda in_seq=0 out_seq=0 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f487eff5640 10 client.15114 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f487eff5640 1 -- 172.21.15.129:0/2835068722 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558076c390e0 con 0x558076dd9000 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f487eff5640 10 client.15114 handle_command_reply: tid=0 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 client.15114 shutdown 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 msgr2=0x558076d531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.162 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f488c0cd190 tx=0x7f488c0094a0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 shutdown_connections 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558076dd9000 0x558076dd7d10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558076d59bb0 0x558076d72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558076d4f2f0 0x558076d536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 --2- 172.21.15.129:0/2835068722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558076d4e9d0 0x558076d531b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 >> 172.21.15.129:0/2835068722 conn(0x558076d4af60 msgr2=0x558076d50c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.156+0000 7f487eff5640 0 client.15114 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 shutdown_connections 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a349c740 1 -- 172.21.15.129:0/2835068722 wait complete. 2024-09-05T01:14:58.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a349c740 20 client.15114 trim_cache size 0 max 16384 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24488, 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 111.270456856, 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-09-05T01:14:58.164 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:14:58.165 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 122.12351779799999 2024-09-05T01:14:58.165 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:14:58.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 msgr2=0x7f48840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 0x7f48840a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f48940096c0 tx=0x7f4894031400 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 msgr2=0x7f489c198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f488c00eab0 tx=0x7f488c00ef70 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 shutdown_connections 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48840a2c10 0x7f48840a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f489c106f20 0x7f489c19cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f489c1027a0 0x7f489c198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 --2- 172.21.15.129:0/2817773300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f489c101df0 0x7f489c198300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 >> 172.21.15.129:0/2817773300 conn(0x7f489c0fd8f0 msgr2=0x7f489c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 shutdown_connections 2024-09-05T01:14:58.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:58.157+0000 7f48a10c3640 1 -- 172.21.15.129:0/2817773300 wait complete. 2024-09-05T01:14:58.905 INFO:tasks.cephadm:Setup complete, yielding 2024-09-05T01:14:58.905 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T01:14:58.915 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:14:58.916 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch status' 2024-09-05T01:14:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:14:58 smithi183 ceph-mon[34589]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:59.163 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:14:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:14:58 smithi129 ceph-mon[30979]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:14:58 smithi194 ceph-mon[34941]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:14:59.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.624+0000 7f4a147b5640 1 -- 172.21.15.129:0/2982740920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c1078c0 msgr2=0x7f4a0c109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:59.646 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.624+0000 7f4a147b5640 1 --2- 172.21.15.129:0/2982740920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c109d10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009ca0 tx=0x7f4a0402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- 172.21.15.129:0/2982740920 shutdown_connections 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- 172.21.15.129:0/2982740920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c109d10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- 172.21.15.129:0/2982740920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 0x7f4a0c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- 172.21.15.129:0/2982740920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c102660 0x7f4a0c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- 172.21.15.129:0/2982740920 >> 172.21.15.129:0/2982740920 conn(0x7f4a0c0fc280 msgr2=0x7f4a0c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- 172.21.15.129:0/2982740920 shutdown_connections 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- 172.21.15.129:0/2982740920 wait complete. 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 Processor -- start 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- start start 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 0x7f4a0c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:59.647 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10d710 con 0x7f4a0c102660 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10d590 con 0x7f4a0c104f90 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a147b5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c10d890 con 0x7f4a0c1078c0 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.625+0000 7f4a12551640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46370/0 (socket says 172.21.15.129:46370) 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 -- 172.21.15.129:0/408061927 learned_addr learned my addr 172.21.15.129:0/408061927 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a11d50640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 0x7f4a0c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12d52640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 -- 172.21.15.129:0/408061927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 msgr2=0x7f4a0c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c100b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 -- 172.21.15.129:0/408061927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 msgr2=0x7f4a0c100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 0x7f4a0c100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.648 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 -- 172.21.15.129:0/408061927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c101410 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12d52640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a12551640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d950 tx=0x7f4a0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08017030 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a08004020 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c1a1f80 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080047a0 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.626+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a2370 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.627+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a0c0fdd70 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.628+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f4a080041d0 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.629+0000 7f4a037fe640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 0x7f49f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.629+0000 7f4a11d50640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 0x7f49f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.629+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f4a080cf6d0 con 0x7f4a0c102660 2024-09-05T01:14:59.649 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.629+0000 7f4a11d50640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 0x7f49f80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c101d40 tx=0x7f49fc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:14:59.650 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.632+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a0809c170 con 0x7f4a0c102660 2024-09-05T01:14:59.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.814+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4a0c062630 con 0x7f49f80a2c10 2024-09-05T01:14:59.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.816+0000 7f4a037fe640 1 -- 172.21.15.129:0/408061927 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4a0c062630 con 0x7f49f80a2c10 2024-09-05T01:14:59.816 INFO:teuthology.orchestra.run.smithi129.stdout:Backend: cephadm 2024-09-05T01:14:59.817 INFO:teuthology.orchestra.run.smithi129.stdout:Available: Yes 2024-09-05T01:14:59.817 INFO:teuthology.orchestra.run.smithi129.stdout:Paused: No 2024-09-05T01:14:59.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 msgr2=0x7f49f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:59.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 0x7f49f80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c101d40 tx=0x7f49fc00b390 comp rx=0 tx=0).stop 2024-09-05T01:14:59.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 msgr2=0x7f4a0c101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:14:59.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d950 tx=0x7f4a0800de10 comp rx=0 tx=0).stop 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 shutdown_connections 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f49f80a2c10 0x7f49f80a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a0c1078c0 0x7f4a0c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a0c104f90 0x7f4a0c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 --2- 172.21.15.129:0/408061927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a0c102660 0x7f4a0c101fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 >> 172.21.15.129:0/408061927 conn(0x7f4a0c0fc280 msgr2=0x7f4a0c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 shutdown_connections 2024-09-05T01:14:59.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:14:59.818+0000 7f4a147b5640 1 -- 172.21.15.129:0/408061927 wait complete. 2024-09-05T01:15:00.596 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-05T01:15:00.841 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:01.107 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:00 smithi129 ceph-mon[30979]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:01.107 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:00 smithi129 ceph-mon[30979]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:00 smithi183 ceph-mon[34589]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:01.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:00 smithi183 ceph-mon[34589]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:00 smithi194 ceph-mon[34941]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:00 smithi194 ceph-mon[34941]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:01.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/997003942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f41017a0 msgr2=0x7fa5f4109040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:01.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/997003942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f41017a0 0x7fa5f4109040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec009ca0 tx=0x7fa5ec02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/997003942 shutdown_connections 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/997003942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f41017a0 0x7fa5f4109040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/997003942 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f4100de0 0x7fa5f4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/997003942 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f41024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.317+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/997003942 >> 172.21.15.129:0/997003942 conn(0x7fa5f40fba40 msgr2=0x7fa5f40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/997003942 shutdown_connections 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/997003942 wait complete. 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 Processor -- start 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- start start 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f41017a0 0x7fa5f4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:01.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410c630 con 0x7fa5f4100de0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410c4b0 con 0x7fa5f41020a0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5fc1ef640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f410c7b0 con 0x7fa5f41017a0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5f9f8b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5f9f8b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46382/0 (socket says 172.21.15.129:46382) 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.318+0000 7fa5f9f8b640 1 -- 172.21.15.129:0/1145579640 learned_addr learned my addr 172.21.15.129:0/1145579640 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5fa78c640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f978a640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f41017a0 0x7fa5f4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 -- 172.21.15.129:0/1145579640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f41017a0 msgr2=0x7fa5f4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f41017a0 0x7fa5f4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 -- 172.21.15.129:0/1145579640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 msgr2=0x7fa5f419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:01.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f419f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 -- 172.21.15.129:0/1145579640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f419fa70 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5fa78c640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5f9f8b640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000d770 tx=0x7fa5f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0017070 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f419fd60 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5f0009c30 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f41a0210 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.319+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0003d00 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.320+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5f41a0500 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.321+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa5f0003ea0 con 0x7fa5f4100de0 2024-09-05T01:15:01.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.322+0000 7fa5eaffd640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 0x7fa5e00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:01.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.322+0000 7fa5f978a640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 0x7fa5e00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:01.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.322+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa5f00cf230 con 0x7fa5f4100de0 2024-09-05T01:15:01.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.322+0000 7fa5f978a640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 0x7fa5e00a5100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4005d00 tx=0x7fa5e4005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:01.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.326+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5f009bcd0 con 0x7fa5f4100de0 2024-09-05T01:15:01.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.501+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa5f4100490 con 0x7fa5e00a2c40 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.513+0000 7fa5eaffd640 1 -- 172.21.15.129:0/1145579640 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7fa5f4100490 con 0x7fa5e00a2c40 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.a smithi129 running (2m) 57s ago 2m 15.7M - 19.1.1-259-g91c8a7e5 77770cf7e895 f96e531a7bca 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.b smithi183 running (2m) 59s ago 2m 28.9M - 19.1.1-259-g91c8a7e5 77770cf7e895 78616fca170d 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.c smithi194 running (2m) 59s ago 2m 16.3M - 19.1.1-259-g91c8a7e5 77770cf7e895 ee8363500991 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.d smithi129 running (2m) 57s ago 2m 14.4M - 19.1.1-259-g91c8a7e5 77770cf7e895 4ad15dc01ebe 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.e smithi183 running (2m) 59s ago 2m 17.7M - 19.1.1-259-g91c8a7e5 77770cf7e895 93726fe80c75 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.f smithi194 running (2m) 59s ago 2m 17.6M - 19.1.1-259-g91c8a7e5 77770cf7e895 f6598336e447 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.g smithi129 running (2m) 57s ago 2m 16.5M - 19.1.1-259-g91c8a7e5 77770cf7e895 47e93a50d053 2024-09-05T01:15:01.514 INFO:teuthology.orchestra.run.smithi129.stdout:mds.h smithi183 running (2m) 59s ago 2m 19.8M - 19.1.1-259-g91c8a7e5 77770cf7e895 00039077e679 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mds.i smithi194 running (2m) 59s ago 2m 17.1M - 19.1.1-259-g91c8a7e5 77770cf7e895 ab1fbeabe146 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mds.j smithi129 running (2m) 57s ago 2m 13.3M - 19.1.1-259-g91c8a7e5 77770cf7e895 efa435e5ea7f 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mds.k smithi183 running (2m) 59s ago 2m 16.9M - 19.1.1-259-g91c8a7e5 77770cf7e895 498078ee559a 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mds.l smithi194 running (2m) 59s ago 2m 13.5M - 19.1.1-259-g91c8a7e5 77770cf7e895 5c07347ae779 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.x smithi129 *:9283,8765 running (9m) 57s ago 9m 507M - 19.1.1-259-g91c8a7e5 77770cf7e895 5ac36ba88b1d 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.y smithi183 *:8443,8765 running (7m) 59s ago 7m 442M - 19.1.1-259-g91c8a7e5 77770cf7e895 a09de669900d 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mgr.z smithi194 *:8443,8765 running (7m) 59s ago 7m 442M - 19.1.1-259-g91c8a7e5 77770cf7e895 4325e8e658f8 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mon.a smithi129 running (9m) 57s ago 9m 53.5M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 ea8dbedddb59 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mon.b smithi183 running (8m) 59s ago 8m 41.2M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 6b76040b4189 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:mon.c smithi194 running (8m) 59s ago 8m 39.4M 2048M 19.1.1-259-g91c8a7e5 77770cf7e895 028c365edf14 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 smithi129 running (7m) 57s ago 7m 47.0M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 559f496a6843 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 smithi183 running (7m) 59s ago 7m 46.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 0151a377b466 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 smithi194 running (6m) 59s ago 6m 54.2M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 c29fa8f4e1e6 2024-09-05T01:15:01.515 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 smithi129 running (6m) 57s ago 6m 55.1M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 f488d543835d 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4 smithi183 running (5m) 59s ago 5m 42.4M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 480cfa2589cc 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5 smithi194 running (5m) 59s ago 5m 53.9M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 41b5e1e3f123 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6 smithi129 running (5m) 57s ago 5m 51.7M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 30d225fc91e4 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7 smithi183 running (4m) 59s ago 4m 52.4M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 06485557bfea 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.8 smithi194 running (4m) 59s ago 4m 50.5M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 ae8448de07fe 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.9 smithi129 running (3m) 57s ago 3m 43.0M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 ec662db70147 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.10 smithi183 running (3m) 59s ago 3m 52.0M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 c26550dc7e00 2024-09-05T01:15:01.516 INFO:teuthology.orchestra.run.smithi129.stdout:osd.11 smithi194 running (2m) 59s ago 2m 50.2M 4272M 19.1.1-259-g91c8a7e5 77770cf7e895 48dc2dc829c9 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 msgr2=0x7fa5e00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 0x7fa5e00a5100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4005d00 tx=0x7fa5e4005c50 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 msgr2=0x7fa5f4197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000d770 tx=0x7fa5f000dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 shutdown_connections 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa5e00a2c40 0x7fa5e00a5100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa5f41020a0 0x7fa5f419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa5f41017a0 0x7fa5f4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 --2- 172.21.15.129:0/1145579640 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa5f4100de0 0x7fa5f4197e40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:01.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 >> 172.21.15.129:0/1145579640 conn(0x7fa5f40fba40 msgr2=0x7fa5f40ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:01.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 shutdown_connections 2024-09-05T01:15:01.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:01.515+0000 7fa5fc1ef640 1 -- 172.21.15.129:0/1145579640 wait complete. 2024-09-05T01:15:02.392 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-05T01:15:02.641 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:03.057 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:02 smithi129 ceph-mon[30979]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:03.057 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:02 smithi129 ceph-mon[30979]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 -- 172.21.15.129:0/1271793586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 msgr2=0x7fe53c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 --2- 172.21.15.129:0/1271793586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c102000 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009900 tx=0x7fe52c02a410 comp rx=0 tx=0).stop 2024-09-05T01:15:03.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 -- 172.21.15.129:0/1271793586 shutdown_connections 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 --2- 172.21.15.129:0/1271793586 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 0x7fe53c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 --2- 172.21.15.129:0/1271793586 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 0x7fe53c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 --2- 172.21.15.129:0/1271793586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c102000 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 -- 172.21.15.129:0/1271793586 >> 172.21.15.129:0/1271793586 conn(0x7fe53c0fd390 msgr2=0x7fe53c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 -- 172.21.15.129:0/1271793586 shutdown_connections 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.135+0000 7fe5415d7640 1 -- 172.21.15.129:0/1271793586 wait complete. 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 Processor -- start 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 -- start start 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:03.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 0x7fe53c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 0x7fe53c19f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d810 con 0x7fe53c101c00 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d690 con 0x7fe53c1037c0 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe5415d7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c10d990 con 0x7fe53c102e00 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe53affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.136+0000 7fe53affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36482/0 (socket says 172.21.15.129:36482) 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 -- 172.21.15.129:0/2805046536 learned_addr learned my addr 172.21.15.129:0/2805046536 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53a7fc640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 0x7fe53c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53b7fe640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 0x7fe53c19f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:03.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 -- 172.21.15.129:0/2805046536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 msgr2=0x7fe53c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 0x7fe53c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 -- 172.21.15.129:0/2805046536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 msgr2=0x7fe53c19f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 0x7fe53c19f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 -- 172.21.15.129:0/2805046536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c19fbc0 con 0x7fe53c101c00 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe53affd640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009230 tx=0x7fe52c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c03f030 con 0x7fe53c101c00 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c19fe50 con 0x7fe53c101c00 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe52c003cb0 con 0x7fe53c101c00 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c035070 con 0x7fe53c101c00 2024-09-05T01:15:03.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.137+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1a0320 con 0x7fe53c101c00 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.138+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c109140 con 0x7fe53c101c00 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.139+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe52c042030 con 0x7fe53c101c00 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.140+0000 7fe523fff640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 0x7fe5240a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.140+0000 7fe53a7fc640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 0x7fe5240a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.140+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe52c0f0020 con 0x7fe53c101c00 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.140+0000 7fe53a7fc640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 0x7fe5240a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe53c1994b0 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:03.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.143+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe52c0bcac0 con 0x7fe53c101c00 2024-09-05T01:15:03.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:02 smithi183 ceph-mon[34589]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:03.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:02 smithi183 ceph-mon[34589]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:02 smithi194 ceph-mon[34941]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:02 smithi194 ceph-mon[34941]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.318+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe53c062630 con 0x7fe5240a2c40 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.324+0000 7fe523fff640 1 -- 172.21.15.129:0/2805046536 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe53c062630 con 0x7fe5240a2c40 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stdout:mds.all 12/12 61s ago 2m smithi129=a;smithi129=d;smithi129=g;smithi129=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi194=c;smithi194=f;smithi194=i;smithi194=l;count:12 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stdout:mgr 3/3 61s ago 7m smithi129=x;smithi183=y;smithi194=z;count:3 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stdout:mon 3/3 61s ago 8m smithi129:172.21.15.129=a;smithi183:172.21.15.183=b;smithi194:172.21.15.194=c;count:3 2024-09-05T01:15:03.326 INFO:teuthology.orchestra.run.smithi129.stdout:osd 12 61s ago - 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 msgr2=0x7fe5240a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 0x7fe5240a5100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe53c1994b0 tx=0x7fe530008040 comp rx=0 tx=0).stop 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 msgr2=0x7fe53c197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe52c009230 tx=0x7fe52c004340 comp rx=0 tx=0).stop 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 shutdown_connections 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe5240a2c40 0x7fe5240a5100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.328 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe53c1037c0 0x7fe53c19f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe53c102e00 0x7fe53c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 --2- 172.21.15.129:0/2805046536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe53c101c00 0x7fe53c197f90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.326+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 >> 172.21.15.129:0/2805046536 conn(0x7fe53c0fd390 msgr2=0x7fe53c106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.327+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 shutdown_connections 2024-09-05T01:15:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:03.327+0000 7fe5415d7640 1 -- 172.21.15.129:0/2805046536 wait complete. 2024-09-05T01:15:04.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-05T01:15:04.329 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 -- 172.21.15.129:0/3120276544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c100ad0 msgr2=0x7f7e6c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 --2- 172.21.15.129:0/3120276544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c100ed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009900 tx=0x7f7e5c02a410 comp rx=0 tx=0).stop 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 -- 172.21.15.129:0/3120276544 shutdown_connections 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 --2- 172.21.15.129:0/3120276544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c102690 0x7f7e6c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 --2- 172.21.15.129:0/3120276544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 --2- 172.21.15.129:0/3120276544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c100ed0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 -- 172.21.15.129:0/3120276544 >> 172.21.15.129:0/3120276544 conn(0x7f7e6c0fc280 msgr2=0x7f7e6c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 -- 172.21.15.129:0/3120276544 shutdown_connections 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.820+0000 7f7e7216d640 1 -- 172.21.15.129:0/3120276544 wait complete. 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 Processor -- start 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 -- start start 2024-09-05T01:15:04.842 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c102690 0x7f7e6c19f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10c6e0 con 0x7f7e6c101cd0 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10c560 con 0x7f7e6c102690 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e7216d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c10c860 con 0x7f7e6c100ad0 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36496/0 (socket says 172.21.15.129:36496) 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 -- 172.21.15.129:0/118253839 learned_addr learned my addr 172.21.15.129:0/118253839 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6b7fe640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6bfff640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c102690 0x7f7e6c19f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 -- 172.21.15.129:0/118253839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c100ad0 msgr2=0x7f7e6c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:04.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 -- 172.21.15.129:0/118253839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c102690 msgr2=0x7f7e6c19f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c102690 0x7f7e6c19f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.821+0000 7f7e6affd640 1 -- 172.21.15.129:0/118253839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c19fc20 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e6affd640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b4a0 tx=0x7f7e6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60014070 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c19ff10 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e60010ac0 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60017c30 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.822+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1a0440 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.823+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c1a06d0 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.824+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f7e60017070 con 0x7f7e6c101cd0 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.825+0000 7f7e68ff9640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 0x7f7e540a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:04.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.825+0000 7f7e6b7fe640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 0x7f7e540a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:04.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.825+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f7e600ceb80 con 0x7f7e6c101cd0 2024-09-05T01:15:04.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.825+0000 7f7e6b7fe640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 0x7f7e540a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a930 tx=0x7f7e5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:04.845 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:04.828+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e6009b730 con 0x7f7e6c101cd0 2024-09-05T01:15:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:04 smithi129 ceph-mon[30979]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:04.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:04 smithi129 ceph-mon[30979]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:05.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.012+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7e6c1a08a0 con 0x7f7e540a2c40 2024-09-05T01:15:05.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.014+0000 7f7e68ff9640 1 -- 172.21.15.129:0/118253839 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7e6c1a08a0 con 0x7f7e540a2c40 2024-09-05T01:15:05.016 INFO:teuthology.orchestra.run.smithi129.stdout:HOST ADDR LABELS STATUS 2024-09-05T01:15:05.016 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129 172.21.15.129 2024-09-05T01:15:05.016 INFO:teuthology.orchestra.run.smithi129.stdout:smithi183 172.21.15.183 2024-09-05T01:15:05.016 INFO:teuthology.orchestra.run.smithi129.stdout:smithi194 172.21.15.194 2024-09-05T01:15:05.017 INFO:teuthology.orchestra.run.smithi129.stdout:3 hosts in cluster 2024-09-05T01:15:05.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 msgr2=0x7f7e540a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:05.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 0x7f7e540a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c02a930 tx=0x7f7e5c000bc0 comp rx=0 tx=0).stop 2024-09-05T01:15:05.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 msgr2=0x7f7e6c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:05.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7e6000b4a0 tx=0x7f7e6000b960 comp rx=0 tx=0).stop 2024-09-05T01:15:05.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 shutdown_connections 2024-09-05T01:15:05.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7e540a2c40 0x7f7e540a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:05.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e6c102690 0x7f7e6c19f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:05.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7e6c101cd0 0x7f7e6c198530 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:05.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 --2- 172.21.15.129:0/118253839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7e6c100ad0 0x7f7e6c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:05.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 >> 172.21.15.129:0/118253839 conn(0x7f7e6c0fc280 msgr2=0x7f7e6c06a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:05.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 shutdown_connections 2024-09-05T01:15:05.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:05.016+0000 7f7e7216d640 1 -- 172.21.15.129:0/118253839 wait complete. 2024-09-05T01:15:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:04 smithi183 ceph-mon[34589]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:04 smithi183 ceph-mon[34589]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:04 smithi194 ceph-mon[34941]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:05.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:04 smithi194 ceph-mon[34941]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:05.850 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-05T01:15:06.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:05 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:15:06.217 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:05 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:15:06.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:05 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:15:06.761 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:07.147 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:07.147 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:07.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.148 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:06 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:07.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:07.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:06 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:06 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 -- 172.21.15.129:0/3204527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d241026d0 msgr2=0x7f0d24108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 --2- 172.21.15.129:0/3204527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d241026d0 0x7f0d24108f60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009ca0 tx=0x7f0d1c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 -- 172.21.15.129:0/3204527138 shutdown_connections 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 --2- 172.21.15.129:0/3204527138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d241026d0 0x7f0d24108f60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 --2- 172.21.15.129:0/3204527138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24101d10 0x7f0d24102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 --2- 172.21.15.129:0/3204527138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d24100b10 0x7f0d24100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 -- 172.21.15.129:0/3204527138 >> 172.21.15.129:0/3204527138 conn(0x7f0d240fc2a0 msgr2=0x7f0d240fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 -- 172.21.15.129:0/3204527138 shutdown_connections 2024-09-05T01:15:07.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.226+0000 7f0d28fce640 1 -- 172.21.15.129:0/3204527138 wait complete. 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 Processor -- start 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 -- start start 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24100b10 0x7f0d2406b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d241026d0 0x7f0d2419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410c6c0 con 0x7f0d24101d10 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410c540 con 0x7f0d24100b10 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d28fce640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2410c840 con 0x7f0d241026d0 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d19d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:07.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d19d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36522/0 (socket says 172.21.15.129:36522) 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d19d9b640 1 -- 172.21.15.129:0/648498087 learned_addr learned my addr 172.21.15.129:0/648498087 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d2259c640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24100b10 0x7f0d2406b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d22d9d640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d241026d0 0x7f0d2419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.227+0000 7f0d19d9b640 1 -- 172.21.15.129:0/648498087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d241026d0 msgr2=0x7f0d2419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d19d9b640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d241026d0 0x7f0d2419f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d19d9b640 1 -- 172.21.15.129:0/648498087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24100b10 msgr2=0x7f0d2406b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d19d9b640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24100b10 0x7f0d2406b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d19d9b640 1 -- 172.21.15.129:0/648498087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2419fab0 con 0x7f0d24101d10 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d19d9b640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d950 tx=0x7f0d0c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:07.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c017070 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0d0c00de80 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2419fda0 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00f620 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.228+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d241a0250 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.229+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d24107f60 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.230+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0d0c009c30 con 0x7f0d24101d10 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.231+0000 7f0d1bfff640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 0x7f0d100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:07.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.231+0000 7f0d2259c640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 0x7f0d100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:07.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.231+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0d0c0cf160 con 0x7f0d24101d10 2024-09-05T01:15:07.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.231+0000 7f0d2259c640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 0x7f0d100a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d2406b870 tx=0x7f0d14009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:07.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.235+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d0c09bc00 con 0x7f0d24101d10 2024-09-05T01:15:07.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.421+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0d24062630 con 0x7f0d100a2c10 2024-09-05T01:15:07.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.423+0000 7f0d1bfff640 1 -- 172.21.15.129:0/648498087 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0d24062630 con 0x7f0d100a2c10 2024-09-05T01:15:07.425 INFO:teuthology.orchestra.run.smithi129.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-05T01:15:07.425 INFO:teuthology.orchestra.run.smithi129.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T01:15:07.425 INFO:teuthology.orchestra.run.smithi129.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T01:15:07.425 INFO:teuthology.orchestra.run.smithi129.stdout:smithi194 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6240002A400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 msgr2=0x7f0d100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 0x7f0d100a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0d2406b870 tx=0x7f0d14009110 comp rx=0 tx=0).stop 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 msgr2=0x7f0d2419cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00d950 tx=0x7f0d0c00dd50 comp rx=0 tx=0).stop 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 shutdown_connections 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f0d100a2c10 0x7f0d100a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0d241026d0 0x7f0d2419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0d24101d10 0x7f0d2419cba0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 --2- 172.21.15.129:0/648498087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0d24100b10 0x7f0d2406b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:07.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.426+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 >> 172.21.15.129:0/648498087 conn(0x7f0d240fc2a0 msgr2=0x7f0d24105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:07.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.427+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 shutdown_connections 2024-09-05T01:15:07.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:07.427+0000 7f0d28fce640 1 -- 172.21.15.129:0/648498087 wait complete. 2024-09-05T01:15:08.097 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T01:15:08.110 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:15:08.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-05T01:15:08.396 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:08.798 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:08.798 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:08.798 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:08.799 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:08.799 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:15:08.799 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:15:08.799 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:08.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.878+0000 7f2401b67640 1 -- 172.21.15.129:0/733405718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 msgr2=0x7f23fc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.878+0000 7f2401b67640 1 --2- 172.21.15.129:0/733405718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc102000 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009ca0 tx=0x7f23ec02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 -- 172.21.15.129:0/733405718 shutdown_connections 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 --2- 172.21.15.129:0/733405718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 0x7f23fc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 --2- 172.21.15.129:0/733405718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23fc102e00 0x7f23fc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 --2- 172.21.15.129:0/733405718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc102000 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 -- 172.21.15.129:0/733405718 >> 172.21.15.129:0/733405718 conn(0x7f23fc0fd390 msgr2=0x7f23fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 -- 172.21.15.129:0/733405718 shutdown_connections 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 -- 172.21.15.129:0/733405718 wait complete. 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.879+0000 7f2401b67640 1 Processor -- start 2024-09-05T01:15:08.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- start start 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23fc102e00 0x7f23fc0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 0x7f23fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10d790 con 0x7f23fc101c00 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10d610 con 0x7f23fc1037c0 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc10d910 con 0x7f23fc102e00 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36548/0 (socket says 172.21.15.129:36548) 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 -- 172.21.15.129:0/104414491 learned_addr learned my addr 172.21.15.129:0/104414491 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:08.908 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fbfff640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 0x7f23fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 -- 172.21.15.129:0/104414491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23fc102e00 msgr2=0x7f23fc0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23fc102e00 0x7f23fc0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 -- 172.21.15.129:0/104414491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 msgr2=0x7f23fc074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 0x7f23fc074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 -- 172.21.15.129:0/104414491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc0753e0 con 0x7f23fc101c00 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23fb7fe640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23ec003ba0 tx=0x7f23ec003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004040 con 0x7f23fc101c00 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a1fa0 con 0x7f23fc101c00 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23ec004690 con 0x7f23fc101c00 2024-09-05T01:15:08.909 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec030cd0 con 0x7f23fc101c00 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.880+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a2450 con 0x7f23fc101c00 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.881+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc1090a0 con 0x7f23fc101c00 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.883+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f23ec0041e0 con 0x7f23fc101c00 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.883+0000 7f23f8ff9640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 0x7f23e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.883+0000 7f23faffd640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 0x7f23e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.883+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f23ec0f0c20 con 0x7f23fc101c00 2024-09-05T01:15:08.910 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.884+0000 7f23faffd640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 0x7f23e40a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23fc075d10 tx=0x7f23f0006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:08.911 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:08.886+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23ec0bd7d0 con 0x7f23fc101c00 2024-09-05T01:15:09.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:09.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:15:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:15:09.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.087+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f23fc02d990 con 0x7f23fc101c00 2024-09-05T01:15:09.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.088+0000 7f23f8ff9640 1 -- 172.21.15.129:0/104414491 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3100 (secure 0 0 0) 0x7f23ec0bd0d0 con 0x7f23fc101c00 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:e25 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:btime 2024-09-05T01:13:50:854675+0000 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:legacy client fscid: 1 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:Filesystem 'cephfs' (1) 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:fs_name cephfs 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 25 2024-09-05T01:15:09.090 INFO:teuthology.orchestra.run.smithi129.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-05T01:13:06.500509+0000 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-09-05T01:13:50.664418+0000 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:tableserver 0 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:root 0 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:session_timeout 300 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:session_autoclose 300 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:max_file_size 1099511627776 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:max_xattr_size 65536 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:required_client_features {} 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:last_failure 0 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:last_failure_osd_epoch 0 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T01:15:09.091 INFO:teuthology.orchestra.run.smithi129.stdout:max_mds 3 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:in 0,1,2 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:up {0=24488,1=24452,2=24451} 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:failed 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:damaged 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:stopped 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:data_pools [3,4] 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:metadata_pool 2 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:inline_data disabled 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:balancer 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:bal_rank_mask -1 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:standby_count_wanted 0 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:qdb_cluster leader: 24488 members: 24451,24452,24488 2024-09-05T01:15:09.092 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.b{0:24488} state up:active seq 4 addr [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.c{0:24439} state up:standby-replay seq 1 addr [v2:172.21.15.194:6836/979076613,v1:172.21.15.194:6837/979076613] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.h{1:24452} state up:active seq 17 addr [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.f{1:24433} state up:standby-replay seq 1 addr [v2:172.21.15.194:6834/1641200590,v1:172.21.15.194:6835/1641200590] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.i{2:24451} state up:active seq 17 addr [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/3903364483,v1:172.21.15.183:6835/3903364483] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:Standby daemons: 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.129:6834/2733798787,v1:172.21.15.129:6835/2733798787] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.129:6836/1098380818,v1:172.21.15.129:6837/1098380818] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.g{-1:14610} state up:standby seq 1 addr [v2:172.21.15.129:6838/2120045366,v1:172.21.15.129:6839/2120045366] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.093 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.129:6840/3095518263,v1:172.21.15.129:6841/3095518263] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.094 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.194:6832/1551944445,v1:172.21.15.194:6833/1551944445] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.094 INFO:teuthology.orchestra.run.smithi129.stdout:[mds.k{-1:24404} state up:standby seq 1 addr [v2:172.21.15.183:6832/263217116,v1:172.21.15.183:6833/263217116] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T01:15:09.094 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:15:09.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.090+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 msgr2=0x7f23e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.090+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 0x7f23e40a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f23fc075d10 tx=0x7f23f0006c90 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.090+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 msgr2=0x7f23fc075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.090+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f23ec003ba0 tx=0x7f23ec003bd0 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 shutdown_connections 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e40a2c10 0x7f23e40a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f23fc1037c0 0x7f23fc074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f23fc102e00 0x7f23fc0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 --2- 172.21.15.129:0/104414491 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101c00 0x7f23fc075f90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 >> 172.21.15.129:0/104414491 conn(0x7f23fc0fd390 msgr2=0x7f23fc106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 shutdown_connections 2024-09-05T01:15:09.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:09.091+0000 7f2401b67640 1 -- 172.21.15.129:0/104414491 wait complete. 2024-09-05T01:15:09.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:15:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:15:09.764 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-05T01:15:09.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:09 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/104414491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T01:15:10.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:09 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/104414491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T01:15:10.010 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:10.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:09 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/104414491' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.484+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2878015097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 msgr2=0x7fa59c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.484+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2878015097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c108f60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa594009ca0 tx=0x7fa59402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2878015097 shutdown_connections 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2878015097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c108f60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2878015097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 0x7fa59c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2878015097 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 0x7fa59c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2878015097 >> 172.21.15.129:0/2878015097 conn(0x7fa59c0fc2a0 msgr2=0x7fa59c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2878015097 shutdown_connections 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2878015097 wait complete. 2024-09-05T01:15:10.499 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 Processor -- start 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- start start 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 0x7fa59c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 0x7fa59c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10c6c0 con 0x7fa59c1026d0 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10c540 con 0x7fa59c101d10 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a2d3d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa59c10c840 con 0x7fa59c100b10 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a12da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a12da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36564/0 (socket says 172.21.15.129:36564) 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.485+0000 7fa5a12da640 1 -- 172.21.15.129:0/2924051427 learned_addr learned my addr 172.21.15.129:0/2924051427 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa59bfff640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 0x7fa59c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a0ad9640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 0x7fa59c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 -- 172.21.15.129:0/2924051427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 msgr2=0x7fa59c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:10.500 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 0x7fa59c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 -- 172.21.15.129:0/2924051427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 msgr2=0x7fa59c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 0x7fa59c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 -- 172.21.15.129:0/2924051427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59c19fa70 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a12da640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa594009dc0 tx=0x7fa594003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594032ec0 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa59c19fd00 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5940044f0 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594030d40 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.486+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa59c1a01e0 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.487+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59c1a0410 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.488+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa594004040 con 0x7fa59c1026d0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.489+0000 7fa599ffb640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 0x7fa58c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.490+0000 7fa5a0ad9640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 0x7fa58c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:10.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.490+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa5940f0c00 con 0x7fa59c1026d0 2024-09-05T01:15:10.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.490+0000 7fa5a0ad9640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 0x7fa58c0a52a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa59c101b70 tx=0x7fa590008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:10.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.493+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5940bd7b0 con 0x7fa59c1026d0 2024-09-05T01:15:10.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.658+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa59c1a07a0 con 0x7fa59c1026d0 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.659+0000 7fa599ffb640 1 -- 172.21.15.129:0/2924051427 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5620 (secure 0 0 0) 0x7fa5940bd0b0 con 0x7fa59c1026d0 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:epoch 88 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:created 2024-09-05T01:05:32.739177+0000 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:modified 2024-09-05T01:14:18.823732+0000 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T01:15:10.661 INFO:teuthology.orchestra.run.smithi129.stdout:crush_version 27 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:full_ratio 0.95 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:backfillfull_ratio 0.9 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:nearfull_ratio 0.85 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:require_min_compat_client luminous 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:min_compat_client luminous 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:require_osd_release squid 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:stretch_mode_enabled false 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:max_osd 12 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6802/2948481734,v1:172.21.15.129:6803/2948481734] [v2:172.21.15.129:6804/2948481734,v1:172.21.15.129:6805/2948481734] exists,up a8916e3b-e0c4-45b9-84bb-5f527d0f0439 2024-09-05T01:15:10.662 INFO:teuthology.orchestra.run.smithi129.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/63466833,v1:172.21.15.183:6801/63466833] [v2:172.21.15.183:6802/63466833,v1:172.21.15.183:6803/63466833] exists,up b6b99a10-04d7-40ff-8801-2acf7628a478 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6800/968743504,v1:172.21.15.194:6801/968743504] [v2:172.21.15.194:6802/968743504,v1:172.21.15.194:6803/968743504] exists,up 45de925c-c48d-4388-8e8d-3afcfd7c2022 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.3 up in weight 1 up_from 24 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6810/171450732,v1:172.21.15.129:6811/171450732] [v2:172.21.15.129:6812/171450732,v1:172.21.15.129:6813/171450732] exists,up 9db73173-f27a-493d-a9c8-478d042973f5 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.4 up in weight 1 up_from 31 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3612040515,v1:172.21.15.183:6809/3612040515] [v2:172.21.15.183:6810/3612040515,v1:172.21.15.183:6811/3612040515] exists,up d31dce58-e63a-47ed-8940-9db4c09d6d1f 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.5 up in weight 1 up_from 36 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6808/1401637167,v1:172.21.15.194:6809/1401637167] [v2:172.21.15.194:6810/1401637167,v1:172.21.15.194:6811/1401637167] exists,up 3c954db0-fb4f-4beb-99fe-abedf9d7e241 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.6 up in weight 1 up_from 43 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6818/2103992741,v1:172.21.15.129:6819/2103992741] [v2:172.21.15.129:6820/2103992741,v1:172.21.15.129:6821/2103992741] exists,up 0213e2f7-1e56-4c20-9122-cd81c2dc4996 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.7 up in weight 1 up_from 48 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/1669309278,v1:172.21.15.183:6817/1669309278] [v2:172.21.15.183:6818/1669309278,v1:172.21.15.183:6819/1669309278] exists,up fd9aefa5-14e5-492a-8eb0-a0704322665a 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.8 up in weight 1 up_from 55 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6816/307349616,v1:172.21.15.194:6817/307349616] [v2:172.21.15.194:6818/307349616,v1:172.21.15.194:6819/307349616] exists,up f64d8289-4220-4110-b3f0-b17294db4107 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.9 up in weight 1 up_from 62 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6826/1276233090,v1:172.21.15.129:6827/1276233090] [v2:172.21.15.129:6828/1276233090,v1:172.21.15.129:6829/1276233090] exists,up 5d297ed4-17e8-44e6-a4a2-70390bf762ab 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.10 up in weight 1 up_from 67 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/1458641027,v1:172.21.15.183:6825/1458641027] [v2:172.21.15.183:6826/1458641027,v1:172.21.15.183:6827/1458641027] exists,up 2d90543e-16fb-4460-ad34-8e7d09c8c119 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:osd.11 up in weight 1 up_from 74 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.194:6824/3769265743,v1:172.21.15.194:6825/3769265743] [v2:172.21.15.194:6826/3769265743,v1:172.21.15.194:6827/3769265743] exists,up aa637f9f-a4b5-41aa-ab8f-66306b7c6565 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.0 [4,8] 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.6 [3,7] 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.1b [5,9] 2024-09-05T01:15:10.663 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.1d [4,6] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.2c [4,9] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 3.33 [5,9] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 4.8 [5,8] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 4.12 [9,10] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 4.13 [9,8] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:pg_upmap_items 4.18 [9,8] 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6801/3942407634 expires 2024-09-06T01:06:17.664286+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6800/3942407634 expires 2024-09-06T01:06:17.664286+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/665628602 expires 2024-09-06T01:06:17.664286+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/3304384691 expires 2024-09-06T01:06:17.664286+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6800/3490891928 expires 2024-09-06T01:05:54.362223+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/2755277182 expires 2024-09-06T01:05:54.362223+0000 2024-09-05T01:15:10.664 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/4057892175 expires 2024-09-06T01:05:54.362223+0000 2024-09-05T01:15:10.665 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/4214690776 expires 2024-09-06T01:06:17.664286+0000 2024-09-05T01:15:10.665 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:6801/3490891928 expires 2024-09-06T01:05:54.362223+0000 2024-09-05T01:15:10.665 INFO:teuthology.orchestra.run.smithi129.stdout:blocklist 172.21.15.129:0/3403026756 expires 2024-09-06T01:05:54.362223+0000 2024-09-05T01:15:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 msgr2=0x7fa58c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:10.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 0x7fa58c0a52a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa59c101b70 tx=0x7fa590008040 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 msgr2=0x7fa59c19f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa594009dc0 tx=0x7fa594003ba0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 shutdown_connections 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa58c0a2de0 0x7fa58c0a52a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa59c1026d0 0x7fa59c19f400 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa59c101d10 0x7fa59c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 --2- 172.21.15.129:0/2924051427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa59c100b10 0x7fa59c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 >> 172.21.15.129:0/2924051427 conn(0x7fa59c0fc2a0 msgr2=0x7fa59c06a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 shutdown_connections 2024-09-05T01:15:10.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:10.662+0000 7fa5a2d3d640 1 -- 172.21.15.129:0/2924051427 wait complete. 2024-09-05T01:15:10.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:10 smithi129 ceph-mon[30979]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:10.978 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:10 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2924051427' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T01:15:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:10 smithi194 ceph-mon[34941]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:11.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:10 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2924051427' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T01:15:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:10 smithi183 ceph-mon[34589]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:11.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:10 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2924051427' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T01:15:11.305 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-05T01:15:11.320 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:11.566 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:12.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- 172.21.15.129:0/3547670585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980101de0 msgr2=0x7ff9801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:12.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 --2- 172.21.15.129:0/3547670585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980101de0 0x7ff9801021c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff970009900 tx=0x7ff97002a410 comp rx=0 tx=0).stop 2024-09-05T01:15:12.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- 172.21.15.129:0/3547670585 shutdown_connections 2024-09-05T01:15:12.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 --2- 172.21.15.129:0/3547670585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 0x7ff98010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 --2- 172.21.15.129:0/3547670585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980102790 0x7ff9801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 --2- 172.21.15.129:0/3547670585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980101de0 0x7ff9801021c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- 172.21.15.129:0/3547670585 >> 172.21.15.129:0/3547670585 conn(0x7ff9800fd8c0 msgr2=0x7ff9800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- 172.21.15.129:0/3547670585 shutdown_connections 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- 172.21.15.129:0/3547670585 wait complete. 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 Processor -- start 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.027+0000 7ff98671b640 1 -- start start 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980101de0 0x7ff9800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 0x7ff980074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff980111e20 con 0x7ff980102790 2024-09-05T01:15:12.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff980111ca0 con 0x7ff980106f10 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff98671b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff980111fa0 con 0x7ff980101de0 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36578/0 (socket says 172.21.15.129:36578) 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 -- 172.21.15.129:0/2858476527 learned_addr learned my addr 172.21.15.129:0/2858476527 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97ffff640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980101de0 0x7ff9800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff984cb8640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 0x7ff980074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 -- 172.21.15.129:0/2858476527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980101de0 msgr2=0x7ff9800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980101de0 0x7ff9800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 -- 172.21.15.129:0/2858476527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 msgr2=0x7ff980074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 0x7ff980074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 -- 172.21.15.129:0/2858476527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9801aac90 con 0x7ff980102790 2024-09-05T01:15:12.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.028+0000 7ff97f7fe640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d770 tx=0x7ff96800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.029+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968017070 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.029+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9680054d0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.029+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801aaec0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.029+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff968004cf0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.029+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9801ab2c0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.030+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9801032c0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.031+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff968004e90 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.032+0000 7ff97d7fa640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 0x7ff96c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.032+0000 7ff97ffff640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 0x7ff96c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.032+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff9680cf3b0 con 0x7ff980102790 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.032+0000 7ff97ffff640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 0x7ff96c0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff970031440 tx=0x7ff970005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:12.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.035+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff96809bdf0 con 0x7ff980102790 2024-09-05T01:15:12.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.235+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9801021c0 con 0x7ff980102790 2024-09-05T01:15:12.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.236+0000 7ff97d7fa640 1 -- 172.21.15.129:0/2858476527 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7ff9801021c0 con 0x7ff980102790 2024-09-05T01:15:12.237 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:12.237 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:15:12.238 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 msgr2=0x7ff96c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 0x7ff96c0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff970031440 tx=0x7ff970005b50 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 msgr2=0x7ff980074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff96800d770 tx=0x7ff96800dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 shutdown_connections 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff96c0a2c10 0x7ff96c0a50d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff980106f10 0x7ff980074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff980102790 0x7ff980074a30 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 --2- 172.21.15.129:0/2858476527 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff980101de0 0x7ff9800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:12.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 >> 172.21.15.129:0/2858476527 conn(0x7ff9800fd8c0 msgr2=0x7ff9800fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:12.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 shutdown_connections 2024-09-05T01:15:12.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:12.238+0000 7ff98671b640 1 -- 172.21.15.129:0/2858476527 wait complete. 2024-09-05T01:15:12.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:12 smithi129 ceph-mon[30979]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:12.979 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:12 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2858476527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:13.010 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:13.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:12 smithi183 ceph-mon[34589]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:13.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:12 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2858476527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:12 smithi194 ceph-mon[34941]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:12 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2858476527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:13.257 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- 172.21.15.129:0/1337191666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102ef0 msgr2=0x7fc57410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 --2- 172.21.15.129:0/1337191666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102ef0 0x7fc57410da40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc564009ca0 tx=0x7fc56402a950 comp rx=0 tx=0).stop 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- 172.21.15.129:0/1337191666 shutdown_connections 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 --2- 172.21.15.129:0/1337191666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102ef0 0x7fc57410da40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 --2- 172.21.15.129:0/1337191666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102570 0x7fc5741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 --2- 172.21.15.129:0/1337191666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 0x7fc574101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- 172.21.15.129:0/1337191666 >> 172.21.15.129:0/1337191666 conn(0x7fc5740fc7d0 msgr2=0x7fc5740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- 172.21.15.129:0/1337191666 shutdown_connections 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- 172.21.15.129:0/1337191666 wait complete. 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 Processor -- start 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.720+0000 7fc57baeb640 1 -- start start 2024-09-05T01:15:13.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 0x7fc574198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102ef0 0x7fc57419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc574110ea0 con 0x7fc574102570 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc574110d20 con 0x7fc574100e20 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc574111020 con 0x7fc574102ef0 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48930/0 (socket says 172.21.15.129:48930) 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 -- 172.21.15.129:0/2771034364 learned_addr learned my addr 172.21.15.129:0/2771034364 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579887640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 0x7fc574198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 -- 172.21.15.129:0/2771034364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102ef0 msgr2=0x7fc57419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57a088640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102ef0 0x7fc57419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:13.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102ef0 0x7fc57419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 -- 172.21.15.129:0/2771034364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 msgr2=0x7fc574198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 0x7fc574198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 -- 172.21.15.129:0/2771034364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc57419d3e0 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc579086640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c920 tx=0x7fc56800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568007bb0 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc568007d50 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc57419d6d0 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.721+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568018730 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.722+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5741a4fb0 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.723+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc574101200 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.724+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc56800f450 con 0x7fc574102570 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.725+0000 7fc572ffd640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 0x7fc55c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:13.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.725+0000 7fc579887640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 0x7fc55c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.725+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fc5680cf8d0 con 0x7fc574102570 2024-09-05T01:15:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.725+0000 7fc579887640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 0x7fc55c0a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009990 tx=0x7fc56c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:13.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.728+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc568014030 con 0x7fc574102570 2024-09-05T01:15:13.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.934+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc57410a0a0 con 0x7fc574102570 2024-09-05T01:15:13.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.934+0000 7fc572ffd640 1 -- 172.21.15.129:0/2771034364 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11930 (secure 0 0 0) 0x7fc56800de20 con 0x7fc574102570 2024-09-05T01:15:13.935 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:13.935 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":25,"btime":"2024-09-05T01:13:50:854675+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:13:50.664418+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:15:13.935 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 25 2024-09-05T01:15:13.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 msgr2=0x7fc55c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:13.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 0x7fc55c0a5150 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc56c009990 tx=0x7fc56c009110 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 msgr2=0x7fc574198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc56800c920 tx=0x7fc56800cd20 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 shutdown_connections 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc55c0a2c90 0x7fc55c0a5150 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc574102ef0 0x7fc57419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc574102570 0x7fc574198970 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 --2- 172.21.15.129:0/2771034364 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc574100e20 0x7fc574198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 >> 172.21.15.129:0/2771034364 conn(0x7fc5740fc7d0 msgr2=0x7fc57406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 shutdown_connections 2024-09-05T01:15:13.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:13.937+0000 7fc57baeb640 1 -- 172.21.15.129:0/2771034364 wait complete. 2024-09-05T01:15:14.616 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:15:14.882 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:15.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:14 smithi129 ceph-mon[30979]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:15.115 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:14 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2771034364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:14 smithi183 ceph-mon[34589]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:15.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:14 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2771034364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:14 smithi194 ceph-mon[34941]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-05T01:15:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:14 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2771034364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:15.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 -- 172.21.15.129:0/4011493807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01101036e0 msgr2=0x7f0110107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:15.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 --2- 172.21.15.129:0/4011493807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01101036e0 0x7f0110107b90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0104009930 tx=0x7f010402a420 comp rx=0 tx=0).stop 2024-09-05T01:15:15.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 -- 172.21.15.129:0/4011493807 shutdown_connections 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 --2- 172.21.15.129:0/4011493807 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 --2- 172.21.15.129:0/4011493807 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01101036e0 0x7f0110107b90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 --2- 172.21.15.129:0/4011493807 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0110102d30 0x7f0110103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 -- 172.21.15.129:0/4011493807 >> 172.21.15.129:0/4011493807 conn(0x7f01100fc7d0 msgr2=0x7f01100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 -- 172.21.15.129:0/4011493807 shutdown_connections 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.360+0000 7f0116914640 1 -- 172.21.15.129:0/4011493807 wait complete. 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 Processor -- start 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 -- start start 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 0x7f0110198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0110111f10 con 0x7f0110102d30 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0110111d90 con 0x7f01101036e0 2024-09-05T01:15:15.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f0116914640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0110112090 con 0x7f01101082a0 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f010ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f010ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48944/0 (socket says 172.21.15.129:48944) 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.361+0000 7f010ffff640 1 -- 172.21.15.129:0/499028348 learned_addr learned my addr 172.21.15.129:0/499028348 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f0114eb1640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010f7fe640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 0x7f0110198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 -- 172.21.15.129:0/499028348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 msgr2=0x7f011019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 -- 172.21.15.129:0/499028348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 msgr2=0x7f0110198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 0x7f0110198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 -- 172.21.15.129:0/499028348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011019d3a0 con 0x7f0110102d30 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f0114eb1640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010f7fe640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 0x7f0110198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010ffff640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f010000c920 tx=0x7f010000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:15.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100007c70 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011019d690 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0100007e10 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100018820 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.362+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a4f70 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.363+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0110103110 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.364+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f01000189c0 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.365+0000 7f010d7fa640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.365+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f01000cf9d0 con 0x7f0110102d30 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.365+0000 7f010f7fe640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.366+0000 7f010f7fe640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 0x7f00f80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f010402a940 tx=0x7f0104031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:15.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.368+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0100014030 con 0x7f0110102d30 2024-09-05T01:15:15.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.544+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0110199750 con 0x7f0110102d30 2024-09-05T01:15:15.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.545+0000 7f010d7fa640 1 -- 172.21.15.129:0/499028348 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24729 (secure 0 0 0) 0x7f010009c500 con 0x7f0110102d30 2024-09-05T01:15:15.546 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:15.546 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":88,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:14:18.823732+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.547+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 msgr2=0x7f00f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.547+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 0x7f00f80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f010402a940 tx=0x7f0104031400 comp rx=0 tx=0).stop 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.547+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 msgr2=0x7f01101983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.547+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f010000c920 tx=0x7f010000cde0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 shutdown_connections 2024-09-05T01:15:15.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f00f80a2c10 0x7f00f80a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f01101082a0 0x7f011019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f01101036e0 0x7f0110198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 --2- 172.21.15.129:0/499028348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0110102d30 0x7f01101983f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 >> 172.21.15.129:0/499028348 conn(0x7f01100fc7d0 msgr2=0x7f0110105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 shutdown_connections 2024-09-05T01:15:15.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:15.548+0000 7f0116914640 1 -- 172.21.15.129:0/499028348 wait complete. 2024-09-05T01:15:16.029 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:15 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/499028348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:16.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:15 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/499028348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:16.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T01:13:06.500509+0000', 'modified': '2024-09-05T01:13:50.664418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24452, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3903364483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3903364483}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3903364483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/1641200590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 1641200590}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 1641200590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/979076613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 979076613}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 979076613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.194:6839/203556383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 203556383}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 203556383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.183:6837/685319691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 685319691}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 685319691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/3533858614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3533858614}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3533858614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24451, 24452, 24488]} 2024-09-05T01:15:16.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-05T01:15:16.237 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T01:15:16.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:15 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/499028348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:16.486 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:16.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/3764051578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4102790 msgr2=0x7fb7c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/3764051578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4102790 0x7fb7c41067e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002a410 comp rx=0 tx=0).stop 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/3764051578 shutdown_connections 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/3764051578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4106f10 0x7fb7c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/3764051578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4102790 0x7fb7c41067e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/3764051578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4101de0 0x7fb7c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/3764051578 >> 172.21.15.129:0/3764051578 conn(0x7fb7c40fd8c0 msgr2=0x7fb7c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/3764051578 shutdown_connections 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.925+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/3764051578 wait complete. 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 Processor -- start 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 -- start start 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4101de0 0x7fb7c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 0x7fb7c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4111e20 con 0x7fb7c4106f10 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4111ca0 con 0x7fb7c4101de0 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cc6a7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c4111fa0 con 0x7fb7c4102790 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cac44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cac44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48966/0 (socket says 172.21.15.129:48966) 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.926+0000 7fb7cac44640 1 -- 172.21.15.129:0/2333449887 learned_addr learned my addr 172.21.15.129:0/2333449887 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 -- 172.21.15.129:0/2333449887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 msgr2=0x7fb7c4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 0x7fb7c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7ca443640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4101de0 0x7fb7c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 0x7fb7c4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 -- 172.21.15.129:0/2333449887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4101de0 msgr2=0x7fb7c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4101de0 0x7fb7c40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 -- 172.21.15.129:0/2333449887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c41a6880 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 0x7fb7c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cac44640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00eaa0 tx=0x7fb7bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7b77fe640 1 -- 172.21.15.129:0/2333449887 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc00cdb0 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7b77fe640 1 -- 172.21.15.129:0/2333449887 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7bc005020 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c41a6b70 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7b77fe640 1 -- 172.21.15.129:0/2333449887 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0116d0 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.927+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c41a70b0 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.929+0000 7fb7b77fe640 1 -- 172.21.15.129:0/2333449887 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb7bc0051d0 con 0x7fb7c4106f10 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 Processor -- start 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 -- start start 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0ded99d0 0x55bd0ded9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bd0de40090 con 0x55bd0dee4bb0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0dde7a70 con 0x55bd0deda2f0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7cea80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bd0de3c050 con 0x55bd0ded99d0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7c9c42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7c9c42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48980/0 (socket says 172.21.15.129:48980) 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7c9c42640 1 -- 172.21.15.129:0/966192980 learned_addr learned my addr 172.21.15.129:0/966192980 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7ca443640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0ded99d0 0x55bd0ded9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.930+0000 7fb7b77fe640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 0x7fb7b00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7cac44640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 0x7fb7b00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 -- 172.21.15.129:0/966192980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0ded99d0 msgr2=0x55bd0ded9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0ded99d0 0x55bd0ded9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 -- 172.21.15.129:0/966192980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 msgr2=0x55bd0dee4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7b77fe640 1 -- 172.21.15.129:0/2333449887 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb7bc0cf730 con 0x7fb7c4106f10 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 -- 172.21.15.129:0/966192980 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0ddf4130 con 0x55bd0dee4bb0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7ca443640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0032f10 tx=0x7fb7c0031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4fa4ea046ad0fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7b4ff9640 1 -- 172.21.15.129:0/966192980 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0045070 con 0x55bd0dee4bb0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7b4ff9640 1 -- 172.21.15.129:0/966192980 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c00035f0 con 0x55bd0dee4bb0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7cac44640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 0x7fb7b00a5150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc097ed0 tx=0x7fb7bc01eb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.931+0000 7fb7b4ff9640 1 -- 172.21.15.129:0/966192980 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0035040 con 0x55bd0dee4bb0 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- 172.21.15.129:0/966192980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 msgr2=0x55bd0dee6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0032f10 tx=0x7fb7c0031400 comp rx=0 tx=0).stop 2024-09-05T01:15:16.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- 172.21.15.129:0/966192980 shutdown_connections 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee6fa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 --2- 172.21.15.129:0/966192980 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0ded99d0 0x55bd0ded9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- 172.21.15.129:0/966192980 >> 172.21.15.129:0/966192980 conn(0x55bd0ded5f60 msgr2=0x55bd0ded6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- 172.21.15.129:0/966192980 shutdown_connections 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- 172.21.15.129:0/966192980 wait complete. 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.932+0000 7fb7cea80740 1 -- start start 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 Processor -- start 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0dee4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bd0dee7780 con 0x55bd0deda2f0 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0ddfaf20 con 0x55bd0ded99d0 2024-09-05T01:15:16.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bd0dee7900 con 0x55bd0dee4bb0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7cea80740 10 client.0 _ll_register_callbacks cb 0x55bd0ddc4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48982/0 (socket says 172.21.15.129:48982) 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 -- 172.21.15.129:0/429226090 learned_addr learned my addr 172.21.15.129:0/429226090 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 -- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 msgr2=0x55bd0dee1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 -- 172.21.15.129:0/429226090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 msgr2=0x55bd0dee4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0dee4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 -- 172.21.15.129:0/429226090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0dbb12b0 con 0x55bd0deda2f0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7ca443640 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009df0 tx=0x7fb7b80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca360e721d13aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.933+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8012860 con 0x55bd0deda2f0 2024-09-05T01:15:16.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb7b8012a00 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c00035f0 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb7b8025050 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 10 client.15186 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd0def3100 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7c00035f0 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b800f040 con 0x55bd0deda2f0 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 10 client.15186 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 20 client.15186 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0def57f0 0x55bd0def6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 4 client.15186 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd0de4d600 con 0x55bd0def57f0 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.934+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0def57f0 0x55bd0def6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.935+0000 7fb7adffb640 10 client.15186 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.935+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0def57f0 0x55bd0def6c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6c16521eaaafe31 server_cookie=e7b6515b4658d51e in_seq=0 out_seq=0 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7adffb640 1 -- 172.21.15.129:0/429226090 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd0de4d600 con 0x55bd0def57f0 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7adffb640 10 client.15186 handle_command_reply: tid=0 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 client.15186 shutdown 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 msgr2=0x55bd0dee1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009df0 tx=0x7fb7b80117b0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 shutdown_connections 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0def57f0 0x55bd0def6c60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dee1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee1760 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 --2- 172.21.15.129:0/429226090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0dee4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7adffb640 0 client.15186 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 >> 172.21.15.129:0/429226090 conn(0x55bd0ded5f60 msgr2=0x55bd0dedbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 shutdown_connections 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 1 -- 172.21.15.129:0/429226090 wait complete. 2024-09-05T01:15:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.936+0000 7fb7cea80740 20 client.15186 trim_cache size 0 max 16384 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 Processor -- start 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 -- start start 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0def5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dedea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def4fa0 con 0x55bd0deda2f0 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def5670 con 0x55bd0ded99d0 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7cea80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def5d40 con 0x55bd0dee4bb0 2024-09-05T01:15:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7ca443640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7ca443640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48986/0 (socket says 172.21.15.129:48986) 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7ca443640 1 -- 172.21.15.129:0/3259351230 learned_addr learned my addr 172.21.15.129:0/3259351230 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.954+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dedea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 -- 172.21.15.129:0/3259351230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 msgr2=0x55bd0dedea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0def5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dedea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 -- 172.21.15.129:0/3259351230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 msgr2=0x55bd0def5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0def5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 -- 172.21.15.129:0/3259351230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0dee2280 con 0x55bd0deda2f0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7ca443640 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb7b801ae90 tx=0x7fb7b8005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3af2444704ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7affff640 1 -- 172.21.15.129:0/3259351230 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8005230 con 0x55bd0deda2f0 2024-09-05T01:15:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7affff640 1 -- 172.21.15.129:0/3259351230 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7b801a3f0 con 0x55bd0deda2f0 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.955+0000 7fb7affff640 1 -- 172.21.15.129:0/3259351230 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80129d0 con 0x55bd0deda2f0 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- 172.21.15.129:0/3259351230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 msgr2=0x55bd0dee4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb7b801ae90 tx=0x7fb7b8005c30 comp rx=0 tx=0).stop 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- 172.21.15.129:0/3259351230 shutdown_connections 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dedea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0deda2f0 0x55bd0dee4410 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- 172.21.15.129:0/3259351230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0ded99d0 0x55bd0def5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- 172.21.15.129:0/3259351230 >> 172.21.15.129:0/3259351230 conn(0x55bd0ded5f60 msgr2=0x55bd0ded6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- 172.21.15.129:0/3259351230 shutdown_connections 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- 172.21.15.129:0/3259351230 wait complete. 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- start start 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 Processor -- start 2024-09-05T01:15:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dede110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dede650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def6ae0 con 0x55bd0ded99d0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def6410 con 0x55bd0deda2f0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bd0def71b0 con 0x55bd0dee4bb0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7cea80740 10 client.0 _ll_register_callbacks cb 0x7fb7b80096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.956+0000 7fb7c9c42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48996/0 (socket says 172.21.15.129:48996) 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 -- 172.21.15.129:0/2102964825 learned_addr learned my addr 172.21.15.129:0/2102964825 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7ca443640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dede110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 -- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 msgr2=0x55bd0dede650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dede650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 -- 172.21.15.129:0/2102964825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 msgr2=0x55bd0dede110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dede110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 -- 172.21.15.129:0/2102964825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0defe120 con 0x55bd0ded99d0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7c9c42640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55bd0ded72e0 tx=0x7fb7c0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a92f5f36269c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0057060 con 0x55bd0ded99d0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb7c003e600 con 0x55bd0ded99d0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b801a3f0 con 0x55bd0ded99d0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb7c005e050 con 0x55bd0ded99d0 2024-09-05T01:15:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7cea80740 10 client.15192 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.957+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bd0deff420 con 0x55bd0ded99d0 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7b801a3f0 con 0x55bd0ded99d0 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb7c0051030 con 0x55bd0ded99d0 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 10 client.15192 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 20 client.15192 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0df63b80 0x55bd0df62890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 4 client.15192 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "status"}) -- 0x55bd0de47b10 con 0x55bd0df63b80 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7ca443640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0df63b80 0x55bd0df62890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7adffb640 10 client.15192 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.958+0000 7fb7ca443640 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0df63b80 0x55bd0df62890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee12c3fdebbb67c3 server_cookie=b3488d5ef3533f8d in_seq=0 out_seq=0 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7adffb640 1 -- 172.21.15.129:0/2102964825 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bd0de47b10 con 0x55bd0df63b80 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7adffb640 10 client.15192 handle_command_reply: tid=0 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 client.15192 shutdown 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 msgr2=0x55bd0deddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55bd0ded72e0 tx=0x7fb7c0032f40 comp rx=0 tx=0).stop 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 shutdown_connections 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55bd0df63b80 0x55bd0df62890 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bd0dee4bb0 0x55bd0dede650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bd0deda2f0 0x55bd0dede110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7adffb640 0 client.15192 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 --2- 172.21.15.129:0/2102964825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bd0ded99d0 0x55bd0deddbd0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 >> 172.21.15.129:0/2102964825 conn(0x55bd0ded5f60 msgr2=0x55bd0dee4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 shutdown_connections 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 1 -- 172.21.15.129:0/2102964825 wait complete. 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.959+0000 7fb7cea80740 20 client.15192 trim_cache size 0 max 16384 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 2, 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24451, 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:16.970 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 89.113765321000002, 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 88, 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 149.31879127600001 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 msgr2=0x7fb7b00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 0x7fb7b00a5150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc097ed0 tx=0x7fb7bc01eb40 comp rx=0 tx=0).stop 2024-09-05T01:15:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 msgr2=0x7fb7c4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00eaa0 tx=0x7fb7bc00ef60 comp rx=0 tx=0).stop 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 shutdown_connections 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb7b00a2c90 0x7fb7b00a5150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7c4106f10 0x7fb7c4074f70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb7c4102790 0x7fb7c4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 --2- 172.21.15.129:0/2333449887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb7c4101de0 0x7fb7c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 >> 172.21.15.129:0/2333449887 conn(0x7fb7c40fd8c0 msgr2=0x7fb7c40fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 shutdown_connections 2024-09-05T01:15:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:16.960+0000 7fb7cc6a7640 1 -- 172.21.15.129:0/2333449887 wait complete. 2024-09-05T01:15:17.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:16 smithi183 ceph-mon[34589]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:16 smithi129 ceph-mon[30979]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:16 smithi194 ceph-mon[34941]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-05T01:15:17.784 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T01:15:17.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:17 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-09-05T01:15:18.043 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T01:15:18.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:17 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-09-05T01:15:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:17 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]: dispatch 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 -- 172.21.15.129:0/3970353820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef481082a0 msgr2=0x7fef4810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 --2- 172.21.15.129:0/3970353820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef481082a0 0x7fef4810ec40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fef38009cd0 tx=0x7fef3802a8e0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 -- 172.21.15.129:0/3970353820 shutdown_connections 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 --2- 172.21.15.129:0/3970353820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef481082a0 0x7fef4810ec40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 --2- 172.21.15.129:0/3970353820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 0x7fef48107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 --2- 172.21.15.129:0/3970353820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef48102d30 0x7fef48103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.544 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.505+0000 7fef501d2640 1 -- 172.21.15.129:0/3970353820 >> 172.21.15.129:0/3970353820 conn(0x7fef480fc7d0 msgr2=0x7fef480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- 172.21.15.129:0/3970353820 shutdown_connections 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- 172.21.15.129:0/3970353820 wait complete. 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 Processor -- start 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- start start 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 0x7fef481989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef481082a0 0x7fef4819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fef48111f50 con 0x7fef48102d30 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef48111dd0 con 0x7fef481036e0 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef501d2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fef481120d0 con 0x7fef481082a0 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.506+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49014/0 (socket says 172.21.15.129:49014) 2024-09-05T01:15:18.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 -- 172.21.15.129:0/2839224847 learned_addr learned my addr 172.21.15.129:0/2839224847 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef481082a0 0x7fef4819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4d76d640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 0x7fef481989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 -- 172.21.15.129:0/2839224847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef481082a0 msgr2=0x7fef4819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef481082a0 0x7fef4819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 -- 172.21.15.129:0/2839224847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 msgr2=0x7fef481989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 0x7fef481989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 -- 172.21.15.129:0/2839224847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4819d450 con 0x7fef48102d30 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef4df6e640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d770 tx=0x7fef4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef46ffd640 1 -- 172.21.15.129:0/2839224847 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef48102d30 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef46ffd640 1 -- 172.21.15.129:0/2839224847 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef400054d0 con 0x7fef48102d30 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4819d740 con 0x7fef48102d30 2024-09-05T01:15:18.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef46ffd640 1 -- 172.21.15.129:0/2839224847 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40004cf0 con 0x7fef48102d30 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.507+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a5020 con 0x7fef48102d30 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.509+0000 7fef46ffd640 1 -- 172.21.15.129:0/2839224847 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fef40004e90 con 0x7fef48102d30 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 Processor -- start 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 -- start start 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef46ffd640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 0x7fef300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 0x7fef300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 0x5596b8f8cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f97670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef46ffd640 1 -- 172.21.15.129:0/2839224847 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9266+0+0 (secure 0 0 0) 0x7fef400cee10 con 0x7fef48102d30 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596b8ef3090 con 0x5596b8f97bb0 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596b8e9aa70 con 0x5596b8f8d2f0 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.510+0000 7fef525ab740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596b8eef050 con 0x5596b8f8c9d0 2024-09-05T01:15:18.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49028/0 (socket says 172.21.15.129:49028) 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 -- 172.21.15.129:0/658528121 learned_addr learned my addr 172.21.15.129:0/658528121 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4d76d640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f97670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4df6e640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 0x5596b8f8cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 0x7fef300a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef3802ae00 tx=0x7fef38031600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 -- 172.21.15.129:0/658528121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 msgr2=0x5596b8f8cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 0x5596b8f8cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 -- 172.21.15.129:0/658528121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 msgr2=0x5596b8f97670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f97670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 -- 172.21.15.129:0/658528121 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b8ea7130 con 0x5596b8f97bb0 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4df6e640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 0x5596b8f8cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4d76d640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:18.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef4e76f640 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fef38009cd0 tx=0x7fef3804b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f25b3930bf94ff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef2ffff640 1 -- 172.21.15.129:0/658528121 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3803d110 con 0x5596b8f97bb0 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef2ffff640 1 -- 172.21.15.129:0/658528121 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef3804f030 con 0x5596b8f97bb0 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.511+0000 7fef2ffff640 1 -- 172.21.15.129:0/658528121 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38030d00 con 0x5596b8f97bb0 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 -- 172.21.15.129:0/658528121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 msgr2=0x5596b8f99fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fef38009cd0 tx=0x7fef3804b000 comp rx=0 tx=0).stop 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 -- 172.21.15.129:0/658528121 shutdown_connections 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f99fa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 --2- 172.21.15.129:0/658528121 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8c9d0 0x5596b8f8cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 -- 172.21.15.129:0/658528121 >> 172.21.15.129:0/658528121 conn(0x5596b8f88f60 msgr2=0x5596b8f89340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 -- 172.21.15.129:0/658528121 shutdown_connections 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.512+0000 7fef525ab740 1 -- 172.21.15.129:0/658528121 wait complete. 2024-09-05T01:15:18.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 -- start start 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 Processor -- start 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f94760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f97bb0 0x5596b8f94ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596b8f9eb90 con 0x5596b8f8c9d0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596b8eadf20 con 0x5596b8f8d2f0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596b8f9ed10 con 0x5596b8f97bb0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef525ab740 10 client.0 _ll_register_callbacks cb 0x5596b8e77030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49034/0 (socket says 172.21.15.129:49034) 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef4df6e640 1 -- 172.21.15.129:0/3240687893 learned_addr learned my addr 172.21.15.129:0/3240687893 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f94760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.513+0000 7fef4e76f640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f97bb0 0x5596b8f94ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 -- 172.21.15.129:0/3240687893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f97bb0 msgr2=0x5596b8f94ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f97bb0 0x5596b8f94ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 -- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 msgr2=0x5596b8f94760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f94760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 -- 172.21.15.129:0/3240687893 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b8c642b0 con 0x5596b8f8c9d0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef40097770 tx=0x7fef4009af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9473b0eb1cfbc8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef400b4060 con 0x5596b8f8c9d0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef3804f030 con 0x5596b8f8c9d0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9266+0+0 (secure 0 0 0) 0x7fef40097980 con 0x5596b8f8c9d0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef40020040 con 0x5596b8f8c9d0 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef525ab740 10 client.15204 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:18.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596b8faa4a0 con 0x5596b8f8c9d0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef3804f030 con 0x5596b8f8c9d0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fef4007e020 con 0x5596b8f8c9d0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.514+0000 7fef525ab740 10 client.15204 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef525ab740 10 client.15204 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef525ab740 20 client.15204 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b8faca00 0x5596b8fade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef525ab740 4 client.15204 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596b8f00600 con 0x5596b8faca00 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b8faca00 0x5596b8fade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef2d7fa640 10 client.15204 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.515+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b8faca00 0x5596b8fade70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3bc48fbadc986ef server_cookie=5e869ea6361a7730 in_seq=0 out_seq=0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3240687893 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596b8f00600 con 0x5596b8faca00 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef2d7fa640 10 client.15204 handle_command_reply: tid=0 2024-09-05T01:15:18.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 client.15204 shutdown 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 msgr2=0x5596b8f973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fef40097770 tx=0x7fef4009af50 comp rx=0 tx=0).stop 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 shutdown_connections 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b8faca00 0x5596b8fade70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f97bb0 0x5596b8f94ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8d2f0 0x5596b8f94760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 --2- 172.21.15.129:0/3240687893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8f973c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef2d7fa640 0 client.15204 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 >> 172.21.15.129:0/3240687893 conn(0x5596b8f88f60 msgr2=0x5596b8f8ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 shutdown_connections 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 1 -- 172.21.15.129:0/3240687893 wait complete. 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.516+0000 7fef525ab740 20 client.15204 trim_cache size 0 max 16384 2024-09-05T01:15:18.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 Processor -- start 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 -- start start 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8c9d0 0x5596b8fade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596b8fac1b0 con 0x5596b8f97bb0 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596b8fac880 con 0x5596b8f8c9d0 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef525ab740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596b8facf50 con 0x5596b8f8d2f0 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49036/0 (socket says 172.21.15.129:49036) 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 -- 172.21.15.129:0/2651221278 learned_addr learned my addr 172.21.15.129:0/2651221278 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4d76d640 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 -- 172.21.15.129:0/2651221278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 msgr2=0x5596b8fab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 -- 172.21.15.129:0/2651221278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8c9d0 msgr2=0x5596b8fade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8c9d0 0x5596b8fade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 -- 172.21.15.129:0/2651221278 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596b8f952a0 con 0x5596b8f97bb0 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4d76d640 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.534+0000 7fef4e76f640 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fef38005fe0 tx=0x7fef38005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2482faea1442ee64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef2f7fe640 1 -- 172.21.15.129:0/2651221278 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef380305a0 con 0x5596b8f97bb0 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef2f7fe640 1 -- 172.21.15.129:0/2651221278 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef38036040 con 0x5596b8f97bb0 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef2f7fe640 1 -- 172.21.15.129:0/2651221278 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38030be0 con 0x5596b8f97bb0 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 -- 172.21.15.129:0/2651221278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 msgr2=0x5596b8f8fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fef38005fe0 tx=0x7fef38005ef0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 -- 172.21.15.129:0/2651221278 shutdown_connections 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f97bb0 0x5596b8f8fcd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 --2- 172.21.15.129:0/2651221278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f8c9d0 0x5596b8fade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 -- 172.21.15.129:0/2651221278 >> 172.21.15.129:0/2651221278 conn(0x5596b8f88f60 msgr2=0x5596b8f8f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 -- 172.21.15.129:0/2651221278 shutdown_connections 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.535+0000 7fef525ab740 1 -- 172.21.15.129:0/2651221278 wait complete. 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 -- start start 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 Processor -- start 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fb68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 0x5596b8faf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596b8fadcf0 con 0x5596b8f8c9d0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5596b8fad620 con 0x5596b8f97bb0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5596b8fae3c0 con 0x5596b8f8d2f0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef525ab740 10 client.0 _ll_register_callbacks cb 0x7fef400056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49038/0 (socket says 172.21.15.129:49038) 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 -- 172.21.15.129:0/3568815496 learned_addr learned my addr 172.21.15.129:0/3568815496 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4e76f640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 0x5596b8faf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fb68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 -- 172.21.15.129:0/3568815496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 msgr2=0x5596b8fb68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fb68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 -- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 msgr2=0x5596b8faf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 0x5596b8faf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4df6e640 1 -- 172.21.15.129:0/3568815496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596b8faff40 con 0x5596b8f8c9d0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4e76f640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 0x5596b8faf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.536+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fb68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef4df6e640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fef4009b250 tx=0x7fef40009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b25952c943590b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef400b4040 con 0x5596b8f8c9d0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9266+0+0 (secure 0 0 0) 0x7fef4009a760 con 0x5596b8f8c9d0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef38036040 con 0x5596b8f8c9d0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef40005180 con 0x5596b8f8c9d0 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 10 client.15210 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:18.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596b8fb4990 con 0x5596b8f8c9d0 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef38036040 con 0x5596b8f8c9d0 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 1, 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24452, 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:18.559 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 91.703895742, 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 89, 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 153.52572850499999 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fef4007e050 con 0x5596b8f8c9d0 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 10 client.15210 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:18.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 10 client.15210 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 20 client.15210 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b901b5f0 0x5596b901a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 4 client.15210 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.537+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "status"}) -- 0x5596b8efab10 con 0x5596b901b5f0 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b901b5f0 0x5596b901a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef4d76d640 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b901b5f0 0x5596b901a300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7718f52432aacfd server_cookie=f495cde7dbaf0a15 in_seq=0 out_seq=0 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef2d7fa640 10 client.15210 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef2d7fa640 1 -- 172.21.15.129:0/3568815496 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5596b8efab10 con 0x5596b901b5f0 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef2d7fa640 10 client.15210 handle_command_reply: tid=0 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 client.15210 shutdown 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 msgr2=0x5596b8fb6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fef4009b250 tx=0x7fef40009e90 comp rx=0 tx=0).stop 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 shutdown_connections 2024-09-05T01:15:18.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x5596b901b5f0 0x5596b901a300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5596b8f97bb0 0x5596b8faf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5596b8f8d2f0 0x5596b8fb68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef2d7fa640 0 client.15210 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 --2- 172.21.15.129:0/3568815496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596b8f8c9d0 0x5596b8fb6380 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 >> 172.21.15.129:0/3568815496 conn(0x5596b8f88f60 msgr2=0x5596b8f8fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 shutdown_connections 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.538+0000 7fef525ab740 1 -- 172.21.15.129:0/3568815496 wait complete. 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef525ab740 20 client.15210 trim_cache size 0 max 16384 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 msgr2=0x7fef300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 0x7fef300a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef3802ae00 tx=0x7fef38031600 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 msgr2=0x7fef481984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d770 tx=0x7fef4000dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 shutdown_connections 2024-09-05T01:15:18.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef300a2b80 0x7fef300a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fef481082a0 0x7fef4819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481036e0 0x7fef481989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 --2- 172.21.15.129:0/2839224847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fef48102d30 0x7fef481984a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 >> 172.21.15.129:0/2839224847 conn(0x7fef480fc7d0 msgr2=0x7fef48105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 shutdown_connections 2024-09-05T01:15:18.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:18.539+0000 7fef501d2640 1 -- 172.21.15.129:0/2839224847 wait complete. 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T01:15:19.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:18 smithi183 ceph-mon[34589]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T01:15:19.190 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:18 smithi129 ceph-mon[30979]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T01:15:19.218 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T01:15:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-05T01:15:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-05T01:15:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-05T01:15:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 6]}]': finished 2024-09-05T01:15:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-05T01:15:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-09-05T01:15:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [5, 11]}]': finished 2024-09-05T01:15:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T01:15:19.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:18 smithi194 ceph-mon[34941]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T01:15:19.465 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- 172.21.15.129:0/66787869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904106f00 msgr2=0x7fb90410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/66787869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904106f00 0x7fb90410ea80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- 172.21.15.129:0/66787869 shutdown_connections 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/66787869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904106f00 0x7fb90410ea80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/66787869 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904102780 0x7fb9041067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/66787869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 0x7fb9041021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- 172.21.15.129:0/66787869 >> 172.21.15.129:0/66787869 conn(0x7fb9040fd8b0 msgr2=0x7fb9040ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- 172.21.15.129:0/66787869 shutdown_connections 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- 172.21.15.129:0/66787869 wait complete. 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 Processor -- start 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.959+0000 7fb90bcb4640 1 -- start start 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 0x7fb9040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904106f00 0x7fb904074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb904111eb0 con 0x7fb904102780 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb904111d30 con 0x7fb904101dd0 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90bcb4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb904112030 con 0x7fb904106f00 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49064/0 (socket says 172.21.15.129:49064) 2024-09-05T01:15:19.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 -- 172.21.15.129:0/1709267436 learned_addr learned my addr 172.21.15.129:0/1709267436 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb909a50640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 0x7fb9040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90a251640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904106f00 0x7fb904074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 -- 172.21.15.129:0/1709267436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904106f00 msgr2=0x7fb904074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904106f00 0x7fb904074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 -- 172.21.15.129:0/1709267436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 msgr2=0x7fb9040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 0x7fb9040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 -- 172.21.15.129:0/1709267436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041a6940 con 0x7fb904102780 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb90924f640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b4a0 tx=0x7fb8f400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.960+0000 7fb8faffd640 1 -- 172.21.15.129:0/1709267436 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4015020 con 0x7fb904102780 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.961+0000 7fb8faffd640 1 -- 172.21.15.129:0/1709267436 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f4011ce0 con 0x7fb904102780 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.961+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9041a6c30 con 0x7fb904102780 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.961+0000 7fb8faffd640 1 -- 172.21.15.129:0/1709267436 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4010dc0 con 0x7fb904102780 2024-09-05T01:15:19.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.961+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041a71c0 con 0x7fb904102780 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb8faffd640 1 -- 172.21.15.129:0/1709267436 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb8f4018020 con 0x7fb904102780 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 Processor -- start 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 -- start start 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ae1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784aec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561784a48090 con 0x561784aecbb0 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5617849efa70 con 0x561784ae22f0 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.963+0000 7fb90e08d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561784a44050 con 0x561784ae19d0 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb8faffd640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 0x7fb8f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.981 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49080/0 (socket says 172.21.15.129:49080) 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 -- 172.21.15.129:0/3105812189 learned_addr learned my addr 172.21.15.129:0/3105812189 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90a251640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ae1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90a251640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 0x7fb8f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb909a50640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784aec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 -- 172.21.15.129:0/3105812189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 msgr2=0x561784ae1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ae1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 -- 172.21.15.129:0/3105812189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 msgr2=0x561784aec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784aec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 -- 172.21.15.129:0/3105812189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617849fc130 con 0x561784aecbb0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb909a50640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784aec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb8faffd640 1 -- 172.21.15.129:0/1709267436 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fb8f40cece0 con 0x7fb904102780 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90924f640 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4059b00 tx=0x7fb8f4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae85e3c4f6446627 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb90a251640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 0x7fb8f00a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005cc0 tx=0x7fb8fc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.982 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb8effff640 1 -- 172.21.15.129:0/3105812189 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4010480 con 0x561784aecbb0 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb8effff640 1 -- 172.21.15.129:0/3105812189 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f409b830 con 0x561784aecbb0 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.964+0000 7fb8effff640 1 -- 172.21.15.129:0/3105812189 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f401f200 con 0x561784aecbb0 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 -- 172.21.15.129:0/3105812189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 msgr2=0x561784aeefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4059b00 tx=0x7fb8f4021000 comp rx=0 tx=0).stop 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 -- 172.21.15.129:0/3105812189 shutdown_connections 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784aeefa0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784aec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3105812189 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ae1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 -- 172.21.15.129:0/3105812189 >> 172.21.15.129:0/3105812189 conn(0x561784addf60 msgr2=0x561784ade340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 -- 172.21.15.129:0/3105812189 shutdown_connections 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.965+0000 7fb90e08d740 1 -- 172.21.15.129:0/3105812189 wait complete. 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 -- start start 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 Processor -- start 2024-09-05T01:15:19.983 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784ae9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784aecbb0 0x561784ae9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561784aef780 con 0x561784ae19d0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561784a02f20 con 0x561784ae22f0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561784aef900 con 0x561784aecbb0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90e08d740 10 client.0 _ll_register_callbacks cb 0x5617849cc030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49084/0 (socket says 172.21.15.129:49084) 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 -- 172.21.15.129:0/3177825842 learned_addr learned my addr 172.21.15.129:0/3177825842 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90a251640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784aecbb0 0x561784ae9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb90924f640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784ae9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 -- 172.21.15.129:0/3177825842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784aecbb0 msgr2=0x561784ae9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.984 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784aecbb0 0x561784ae9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 -- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 msgr2=0x561784ae9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.966+0000 7fb909a50640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784ae9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb909a50640 1 -- 172.21.15.129:0/3177825842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617847b92b0 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb909a50640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb900009f60 tx=0x7fb900009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c4fcb16e429603 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900019070 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fb900015870 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f409b830 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb900025050 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 10 client.15222 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561784afb0b0 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f409b830 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb900014a90 con 0x561784ae19d0 2024-09-05T01:15:19.985 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 10 client.15222 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 20 client.15222 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784afd6f0 0x561784afeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 4 client.15222 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.967+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561784a55600 con 0x561784afd6f0 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.968+0000 7fb90924f640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784afd6f0 0x561784afeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.968+0000 7fb8ed7fa640 10 client.15222 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.968+0000 7fb90924f640 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784afd6f0 0x561784afeb60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd16fb492b3eb305 server_cookie=4073fbd47b658ffd in_seq=0 out_seq=0 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/3177825842 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561784a55600 con 0x561784afd6f0 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb8ed7fa640 10 client.15222 handle_command_reply: tid=0 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 client.15222 shutdown 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 msgr2=0x561784aec400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb900009f60 tx=0x7fb900009f90 comp rx=0 tx=0).stop 2024-09-05T01:15:19.986 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 shutdown_connections 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784afd6f0 0x561784afeb60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784aecbb0 0x561784ae9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784ae9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 --2- 172.21.15.129:0/3177825842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae19d0 0x561784aec400 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb8ed7fa640 0 client.15222 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 >> 172.21.15.129:0/3177825842 conn(0x561784addf60 msgr2=0x561784ae3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 shutdown_connections 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 1 -- 172.21.15.129:0/3177825842 wait complete. 2024-09-05T01:15:19.987 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.969+0000 7fb90e08d740 20 client.15222 trim_cache size 0 max 16384 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 Processor -- start 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 -- start start 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ade340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784aecbb0 0x561784b07390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561784afcea0 con 0x561784ae22f0 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561784afd570 con 0x561784aecbb0 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90e08d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561784afdc40 con 0x561784ae19d0 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49094/0 (socket says 172.21.15.129:49094) 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 -- 172.21.15.129:0/288681075 learned_addr learned my addr 172.21.15.129:0/288681075 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:19.992 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90a251640 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784aecbb0 0x561784b07390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb909a50640 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ade340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 -- 172.21.15.129:0/288681075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 msgr2=0x561784ade340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ade340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 -- 172.21.15.129:0/288681075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784aecbb0 msgr2=0x561784b07390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784aecbb0 0x561784b07390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.987+0000 7fb90924f640 1 -- 172.21.15.129:0/288681075 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561784addd90 con 0x561784ae22f0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90924f640 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f409aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6350c40509d543c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb8ef7fe640 1 -- 172.21.15.129:0/288681075 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b4070 con 0x561784ae22f0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb8ef7fe640 1 -- 172.21.15.129:0/288681075 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f409b830 con 0x561784ae22f0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb8ef7fe640 1 -- 172.21.15.129:0/288681075 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40994c0 con 0x561784ae22f0 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 -- 172.21.15.129:0/288681075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 msgr2=0x561784b06e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4000c00 tx=0x7fb8f409aa90 comp rx=0 tx=0).stop 2024-09-05T01:15:19.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 -- 172.21.15.129:0/288681075 shutdown_connections 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784aecbb0 0x561784b07390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784ae22f0 0x561784b06e50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 --2- 172.21.15.129:0/288681075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784ade340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.988+0000 7fb90e08d740 1 -- 172.21.15.129:0/288681075 >> 172.21.15.129:0/288681075 conn(0x561784addf60 msgr2=0x561784add000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- 172.21.15.129:0/288681075 shutdown_connections 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- 172.21.15.129:0/288681075 wait complete. 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- start start 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 Processor -- start 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784b04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784b04b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561784afe9e0 con 0x561784aecbb0 2024-09-05T01:15:19.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561784afe310 con 0x561784ae22f0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561784aff0b0 con 0x561784ae19d0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90a251640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.989+0000 7fb90e08d740 10 client.0 _ll_register_callbacks cb 0x7fb900009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49110/0 (socket says 172.21.15.129:49110) 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 -- 172.21.15.129:0/1340720880 learned_addr learned my addr 172.21.15.129:0/1340720880 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90924f640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784b04b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb909a50640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784b04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 -- 172.21.15.129:0/1340720880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 msgr2=0x561784b04650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784b04650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 -- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 msgr2=0x561784b04b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784b04b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 -- 172.21.15.129:0/1340720880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561784b5f400 con 0x561784aecbb0 2024-09-05T01:15:19.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90924f640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784b04b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90a251640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb1dd144d8f2d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb8ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc004040 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fb8fc0041e0 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f409b830 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc004b40 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90e08d740 10 client.15228 fetch_fsmap learned FSMap version 25 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.990+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561784b679b0 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8f409b830 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb8fc030920 con 0x561784aecbb0 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 10 client.15228 fetch_fsmap finished waiting for FSMap version 25 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 10 client.15228 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 20 client.15228 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:19.996 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784b6c5e0 0x561784b6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:19.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 4 client.15228 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:19.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "status"}) -- 0x561784a4fb10 con 0x561784b6c5e0 2024-09-05T01:15:19.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb909a50640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784b6c5e0 0x561784b6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:19.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb909a50640 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784b6c5e0 0x561784b6b2f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36ce6e977a7f76a server_cookie=8631ba1dbc9f3a38 in_seq=0 out_seq=0 2024-09-05T01:15:19.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.991+0000 7fb8ed7fa640 10 client.15228 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb8ed7fa640 1 -- 172.21.15.129:0/1340720880 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561784a4fb10 con 0x561784b6c5e0 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb8ed7fa640 10 client.15228 handle_command_reply: tid=0 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 client.15228 shutdown 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 msgr2=0x561784b5edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc009ca0 tx=0x7fb8fc02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 shutdown_connections 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561784b6c5e0 0x561784b6b2f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561784aecbb0 0x561784b5edc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561784ae22f0 0x561784b04b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb8ed7fa640 0 client.15228 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 --2- 172.21.15.129:0/1340720880 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561784ae19d0 0x561784b04650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 >> 172.21.15.129:0/1340720880 conn(0x561784addf60 msgr2=0x561784aeae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 shutdown_connections 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 1 -- 172.21.15.129:0/1340720880 wait complete. 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.992+0000 7fb90e08d740 20 client.15228 trim_cache size 0 max 16384 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-09-05T01:15:19.999 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24488, 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 133.105876115, 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 25, 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 90, 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 86, 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 143.95893694700001 2024-09-05T01:15:20.000 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 msgr2=0x7fb8f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 0x7fb8f00a50c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005cc0 tx=0x7fb8fc0095c0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 msgr2=0x7fb904074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400b4a0 tx=0x7fb8f400b8a0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 shutdown_connections 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb8f00a2c00 0x7fb8f00a50c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb904106f00 0x7fb904074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb904102780 0x7fb904074a30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 --2- 172.21.15.129:0/1709267436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb904101dd0 0x7fb9040744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 >> 172.21.15.129:0/1709267436 conn(0x7fb9040fd8b0 msgr2=0x7fb9040fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:20.001 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 shutdown_connections 2024-09-05T01:15:20.002 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:19.993+0000 7fb90bcb4640 1 -- 172.21.15.129:0/1709267436 wait complete. 2024-09-05T01:15:20.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:19 smithi183 ceph-mon[34589]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T01:15:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:19 smithi129 ceph-mon[30979]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T01:15:20.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:19 smithi194 ceph-mon[34941]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T01:15:20.715 INFO:teuthology.run_tasks:Running task sequential... 2024-09-05T01:15:20.727 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-05T01:15:20.728 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-05T01:15:20.728 INFO:teuthology.task.print:Enabling mgr modules 2024-09-05T01:15:20.728 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-05T01:15:20.737 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-05T01:15:20.738 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-05T01:15:20.738 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-09-05T01:15:20.738 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T01:15:20.738 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-05T01:15:20.738 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-05T01:15:20.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:20.738 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-05T01:15:20.756 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-05T01:15:20.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:20.756 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T01:15:20.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:20.825 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-05T01:15:20.930 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-05T01:15:20.930 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-05T01:15:20.931 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T01:15:20.944 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:20 smithi129 ceph-mon[30979]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-05T01:15:20.944 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:20 smithi129 ceph-mon[30979]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-05T01:15:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:20 smithi183 ceph-mon[34589]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-05T01:15:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:20 smithi183 ceph-mon[34589]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-05T01:15:21.178 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:20 smithi194 ceph-mon[34941]: pgmap v351: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-05T01:15:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:20 smithi194 ceph-mon[34941]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-05T01:15:21.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 -- 172.21.15.129:0/277272253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508102790 msgr2=0x7f25081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:21.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 --2- 172.21.15.129:0/277272253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508102790 0x7f25081067e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02a410 comp rx=0 tx=0).stop 2024-09-05T01:15:21.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 -- 172.21.15.129:0/277272253 shutdown_connections 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 --2- 172.21.15.129:0/277272253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508106f10 0x7f250810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 --2- 172.21.15.129:0/277272253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508102790 0x7f25081067e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 --2- 172.21.15.129:0/277272253 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508101de0 0x7f25081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 -- 172.21.15.129:0/277272253 >> 172.21.15.129:0/277272253 conn(0x7f25080fd8c0 msgr2=0x7f25080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 -- 172.21.15.129:0/277272253 shutdown_connections 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.686+0000 7f251014f640 1 -- 172.21.15.129:0/277272253 wait complete. 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 Processor -- start 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 -- start start 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508102790 0x7f2508074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:21.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508106f10 0x7f2508074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2508111e20 con 0x7f2508101de0 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2508111ca0 con 0x7f2508102790 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f251014f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2508111fa0 con 0x7f2508106f10 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49128/0 (socket says 172.21.15.129:49128) 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 -- 172.21.15.129:0/661859718 learned_addr learned my addr 172.21.15.129:0/661859718 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250d6ea640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508102790 0x7f2508074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250e6ec640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508106f10 0x7f2508074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 -- 172.21.15.129:0/661859718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508106f10 msgr2=0x7f2508074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508106f10 0x7f2508074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 -- 172.21.15.129:0/661859718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508102790 msgr2=0x7f2508074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:21.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508102790 0x7f2508074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.687+0000 7f250deeb640 1 -- 172.21.15.129:0/661859718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25081a6860 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f250deeb640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f250000d950 tx=0x7f250000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500017070 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25081a6b50 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2500009c30 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250000f620 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.688+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081a7060 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.689+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25081021c0 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.690+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f25000109a0 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.690+0000 7f2506ffd640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 0x7f24f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.690+0000 7f250d6ea640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 0x7f24f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.690+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f25000cf080 con 0x7f2508101de0 2024-09-05T01:15:21.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.691+0000 7f250d6ea640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 0x7f24f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24fc031440 tx=0x7f24fc005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:21.701 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.694+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2500009de0 con 0x7f2508101de0 2024-09-05T01:15:21.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.902+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2508109bc0 con 0x7f2508101de0 2024-09-05T01:15:21.903 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.903+0000 7f2506ffd640 1 -- 172.21.15.129:0/661859718 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7f250009bbd0 con 0x7f2508101de0 2024-09-05T01:15:21.904 INFO:teuthology.orchestra.run.smithi129.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 msgr2=0x7f24f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 0x7f24f00a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f24fc031440 tx=0x7f24fc005b50 comp rx=0 tx=0).stop 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 msgr2=0x7f25080744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f250000d950 tx=0x7f250000de10 comp rx=0 tx=0).stop 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 shutdown_connections 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f00a2b80 0x7f24f00a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.906 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2508106f10 0x7f2508074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2508102790 0x7f2508074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 --2- 172.21.15.129:0/661859718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2508101de0 0x7f25080744f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:21.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 >> 172.21.15.129:0/661859718 conn(0x7f25080fd8c0 msgr2=0x7f25080fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:21.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 shutdown_connections 2024-09-05T01:15:21.907 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:21.905+0000 7f251014f640 1 -- 172.21.15.129:0/661859718 wait complete. 2024-09-05T01:15:22.513 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-05T01:15:22.513 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T01:15:22.513 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi129.front.sepia.ceph.com 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T01:15:22.514 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-05T01:15:22.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:22.514 DEBUG:teuthology.orchestra.run.smithi129:> ip addr 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 ::1/128 scope host 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:38 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:15:22.571 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f0 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:d9:19:39 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: altname enp7s0f1 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:60 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: link/ether 0c:c4:7a:88:72:61 brd ff:ff:ff:ff:ff:ff 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: inet 172.21.15.129/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft 42159sec preferred_lft 42159sec 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: inet6 fe80::ec4:7aff:fe88:7261/64 scope link noprefixroute 2024-09-05T01:15:22.572 INFO:teuthology.orchestra.run.smithi129.stdout: valid_lft forever preferred_lft forever 2024-09-05T01:15:22.573 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add name ceph-brx type bridge 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr flush dev ceph-brx 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set ceph-brx up 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T01:15:22.573 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:15:22.660 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:15:22.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:22.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:22.712 DEBUG:teuthology.orchestra.run.smithi129:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T01:15:22.737 INFO:teuthology.orchestra.run.smithi129.stdout:1 2024-09-05T01:15:22.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:22.738 DEBUG:teuthology.orchestra.run.smithi129:> route 2024-09-05T01:15:22.795 INFO:teuthology.orchestra.run.smithi129.stdout:Kernel IP routing table 2024-09-05T01:15:22.795 INFO:teuthology.orchestra.run.smithi129.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T01:15:22.795 INFO:teuthology.orchestra.run.smithi129.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T01:15:22.795 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T01:15:22.795 INFO:teuthology.orchestra.run.smithi129.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T01:15:22.796 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:15:22.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:15:22.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:22.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:22.924 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-05T01:15:22.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:22.939 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list-id 2024-09-05T01:15:22.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:22 smithi129 ceph-mon[30979]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 3 op/s; 144 B/s, 0 objects/s recovering 2024-09-05T01:15:22.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:22 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/661859718' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:15:22.994 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:15:22.994 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:15:22.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-05T01:15:22.994 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-05T01:15:22.995 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:15:23.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:15:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:23.111 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-05T01:15:23.112 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:15:23.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:15:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:22 smithi183 ceph-mon[34589]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 3 op/s; 144 B/s, 0 objects/s recovering 2024-09-05T01:15:23.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:22 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/661859718' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:15:23.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:23.225 DEBUG:teuthology.orchestra.run.smithi129:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T01:15:23.226 DEBUG:teuthology.orchestra.run.smithi129:> set -e 2024-09-05T01:15:23.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set brx.0 up 2024-09-05T01:15:23.226 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T01:15:23.226 DEBUG:teuthology.orchestra.run.smithi129:> ') 2024-09-05T01:15:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:22 smithi194 ceph-mon[34941]: pgmap v352: 129 pgs: 2 peering, 127 active+clean; 642 KiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 639 B/s wr, 3 op/s; 144 B/s, 0 objects/s recovering 2024-09-05T01:15:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:22 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/661859718' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T01:15:23.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T01:15:23.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:23.302 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2024-09-05T01:15:23.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T01:15:23.303 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -v /tmp/cephfs 2024-09-05T01:15:23.318 INFO:teuthology.orchestra.run.smithi129.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-05T01:15:23.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T01:15:23.318 DEBUG:teuthology.orchestra.run.smithi129:> chmod 0000 /tmp/cephfs 2024-09-05T01:15:23.373 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-09-05T01:15:23.441 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-09-05T01:15:23.498 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-09-05T01:15:23.499 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/bc5b16d8b1bb7d929277aeb93eb862115f1162cf050ea5a605472c71523282d1/merged 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a9d8e443514e1b43451c70231c10b54477b248cfb48ee2100dd5954906931016/merged 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/0 2024-09-05T01:15:23.500 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/9e3a8c301999c6a31f1cb49349caa5c281e688a2ff7873f3bc02f5e045ae807c/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a27ef43250c18d4f0de027197eea562db771db73ec4dd5a76ede93f1de32407f/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/76b7e3cc342992e8d2f38e13eeb34d9084309dfefcef60ecbc2ab34f036ac6cf/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/abc3d332c7e32f435eccf9439fd3dc72aaaf6ccf18adab417caf7f5b8de7ddea/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/38493fa094ff6b25db46252e4ada6ebc9ffe48f85e05e75fd9ef24d93242d781/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/e07dcd732625a75fd5ea12a2f6c01c27646a9e50d72e8b1a488036c5490642f7/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/b0948983ff36c2ab972d19716e21d96c48dd67a358dfd822f93154fae5f612b4/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/81f58d290eca11f257b1d7dc4c0be37b6df06a645878a08ac32e720ae5d141b6/merged 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-05T01:15:23.501 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-05T01:15:23.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:23.502 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-09-05T01:15:23.554 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T01:15:23.554 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/qa/sv_0/0fb29cbb-b1e0-4ed3-9e40-e7d25670feba --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2024-09-05T01:15:23.598 DEBUG:teuthology.orchestra.run.smithi129:> sudo modprobe fuse 2024-09-05T01:15:23.623 DEBUG:teuthology.orchestra.run.smithi129:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/proc 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/sys 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/dev 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/security 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/shm 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/pts 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/run 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/cgroup 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/pstore 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/bpf 2024-09-05T01:15:23.680 INFO:teuthology.orchestra.run.smithi129.stdout:/ 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/selinux 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/mqueue 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/dev/hugepages 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/debug 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/tracing 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/kernel/config 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/sys/fs/fuse/connections 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-sysusers.service 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T01:15:23.681 INFO:teuthology.orchestra.run.smithi129.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/1000 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/ceph 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/bc5b16d8b1bb7d929277aeb93eb862115f1162cf050ea5a605472c71523282d1/merged 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a9d8e443514e1b43451c70231c10b54477b248cfb48ee2100dd5954906931016/merged 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/run/user/0 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/9e3a8c301999c6a31f1cb49349caa5c281e688a2ff7873f3bc02f5e045ae807c/merged 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/a27ef43250c18d4f0de027197eea562db771db73ec4dd5a76ede93f1de32407f/merged 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/76b7e3cc342992e8d2f38e13eeb34d9084309dfefcef60ecbc2ab34f036ac6cf/merged 2024-09-05T01:15:23.682 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/abc3d332c7e32f435eccf9439fd3dc72aaaf6ccf18adab417caf7f5b8de7ddea/merged 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/38493fa094ff6b25db46252e4ada6ebc9ffe48f85e05e75fd9ef24d93242d781/merged 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/e07dcd732625a75fd5ea12a2f6c01c27646a9e50d72e8b1a488036c5490642f7/merged 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/b0948983ff36c2ab972d19716e21d96c48dd67a358dfd822f93154fae5f612b4/merged 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/var/lib/containers/storage/overlay/81f58d290eca11f257b1d7dc4c0be37b6df06a645878a08ac32e720ae5d141b6/merged 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run.smithi129.stdout:/run/netns/ceph-ns--tmp-cephfs 2024-09-05T01:15:23.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:23.683 DEBUG:teuthology.orchestra.run.smithi129:> ls /sys/fs/fuse/connections 2024-09-05T01:15:23.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:2024-09-05T01:15:23.681+0000 7ff586916480 -1 init, newargv = 0x562363acaf90 newargc=17 2024-09-05T01:15:23.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[84335]: starting ceph client 2024-09-05T01:15:23.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[84335]: starting fuse 2024-09-05T01:15:23.700 INFO:teuthology.orchestra.run.smithi129.stdout:121 2024-09-05T01:15:23.700 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-09-05T01:15:23.700 DEBUG:teuthology.orchestra.run.smithi129:> sudo stdin-killer -- python3 -c ' 2024-09-05T01:15:23.700 DEBUG:teuthology.orchestra.run.smithi129:> import glob 2024-09-05T01:15:23.700 DEBUG:teuthology.orchestra.run.smithi129:> import re 2024-09-05T01:15:23.700 DEBUG:teuthology.orchestra.run.smithi129:> import os 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> import subprocess 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> def _find_admin_socket(client_name): 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> files = glob.glob(asok_path) 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> mountpoint = "/tmp/cephfs" 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> # Given a non-glob path, it better be there 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> if "*" not in asok_path: 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> assert(len(files) == 1) 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> return files[0] 2024-09-05T01:15:23.701 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> for f in files: 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> contents = proc_f.read() 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> if mountpoint in contents: 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> return f 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> print(_find_admin_socket("client.0")) 2024-09-05T01:15:23.702 DEBUG:teuthology.orchestra.run.smithi129:> ' 2024-09-05T01:15:23.818 INFO:teuthology.orchestra.run.smithi129.stdout:/var/run/ceph/ceph-client.0.84335.asok 2024-09-05T01:15:23.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T01:15:23.828 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.84335.asok 2024-09-05T01:15:23.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:23.829 DEBUG:teuthology.orchestra.run.smithi129:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.84335.asok status 2024-09-05T01:15:23.963 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:23.963 INFO:teuthology.orchestra.run.smithi129.stdout: "metadata": { 2024-09-05T01:15:23.963 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_sha1": "91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4", 2024-09-05T01:15:23.963 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.1.1-259-g91c8a7e5 (91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4) squid (stable)", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "entity_id": "0", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": "smithi129", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "mount_point": "/tmp/cephfs", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "pid": "84335", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "root": "/volumes/qa/sv_0/0fb29cbb-b1e0-4ed3-9e40-e7d25670feba" 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_count": 0, 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "dentry_pinned_count": 0, 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 15240, 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "inst": { 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "name": { 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "client", 2024-09-05T01:15:23.964 INFO:teuthology.orchestra.run.smithi129.stdout: "num": 15240 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 4199543302 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": { 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "192.168.0.1:0", 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 4199543302 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:15:23.965 INFO:teuthology.orchestra.run.smithi129.stdout: "inst_str": "client.15240 192.168.0.1:0/4199543302", 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "addr_str": "192.168.0.1:0/4199543302", 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "inode_count": 5, 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_epoch": 25, 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch": 90, 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_epoch_barrier": 0, 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "blocklisted": false, 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs" 2024-09-05T01:15:23.966 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:23.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:23.972 DEBUG:teuthology.orchestra.run.smithi129:> stat --file-system '--printf=%T 2024-09-05T01:15:23.972 DEBUG:teuthology.orchestra.run.smithi129:> ' -- /tmp/cephfs 2024-09-05T01:15:23.988 INFO:teuthology.orchestra.run.smithi129.stdout:fuseblk 2024-09-05T01:15:23.988 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2024-09-05T01:15:23.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:15:23.989 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 1777 /tmp/cephfs 2024-09-05T01:15:24.051 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:23 smithi129 ceph-mon[30979]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:24.052 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:23 smithi129 ceph-mon[30979]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-05T01:15:24.052 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:23 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:15:24.056 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T01:15:24.065 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T01:15:24.066 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:15:24.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-05T01:15:24.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 -- 172.21.15.129:0/133004936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888100a60 msgr2=0x7f0888100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 --2- 172.21.15.129:0/133004936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888100a60 0x7f0888100e60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0878009900 tx=0x7f087802a410 comp rx=0 tx=0).stop 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 -- 172.21.15.129:0/133004936 shutdown_connections 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 --2- 172.21.15.129:0/133004936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888102610 0x7f0888108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 --2- 172.21.15.129:0/133004936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888101c50 0x7f08881020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 --2- 172.21.15.129:0/133004936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888100a60 0x7f0888100e60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 -- 172.21.15.129:0/133004936 >> 172.21.15.129:0/133004936 conn(0x7f08880fc1c0 msgr2=0x7f08880fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 -- 172.21.15.129:0/133004936 shutdown_connections 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.196+0000 7f088da1e640 1 -- 172.21.15.129:0/133004936 wait complete. 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 Processor -- start 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 -- start start 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888100a60 0x7f088806ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:24.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888102610 0x7f08881a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f088810c590 con 0x7f0888101c50 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f088810c410 con 0x7f0888100a60 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f088da1e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f088810c710 con 0x7f0888102610 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f087ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f087ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52136/0 (socket says 172.21.15.129:52136) 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f087ffff640 1 -- 172.21.15.129:0/835217142 learned_addr learned my addr 172.21.15.129:0/835217142 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f08877fe640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888102610 0x7f08881a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.197+0000 7f0886ffd640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888100a60 0x7f088806ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 -- 172.21.15.129:0/835217142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888102610 msgr2=0x7f08881a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888102610 0x7f08881a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 -- 172.21.15.129:0/835217142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888100a60 msgr2=0x7f088806ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:24.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888100a60 0x7f088806ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 -- 172.21.15.129:0/835217142 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08881a1ba0 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f087ffff640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f087000d770 tx=0x7f087000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870017070 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08700054d0 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08881a1e30 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004cf0 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.198+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a22f0 con 0x7f0888101c50 2024-09-05T01:15:24.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.199+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0888107e70 con 0x7f0888101c50 2024-09-05T01:15:24.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.200+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f0870004e90 con 0x7f0888101c50 2024-09-05T01:15:24.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.201+0000 7f088ca1c640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 0x7f08740a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:24.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.201+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f08700cee20 con 0x7f0888101c50 2024-09-05T01:15:24.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.201+0000 7f0886ffd640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 0x7f08740a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:24.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.201+0000 7f0886ffd640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 0x7f08740a51a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f08780314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:24.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.205+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f087009b970 con 0x7f0888101c50 2024-09-05T01:15:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:23 smithi194 ceph-mon[34941]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:23 smithi194 ceph-mon[34941]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-05T01:15:24.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:23 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:15:24.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:23 smithi183 ceph-mon[34589]: pgmap v353: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 639 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:23 smithi183 ceph-mon[34589]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-05T01:15:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:23 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:15:24.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.410+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f0888062e90 con 0x7f0888101c50 2024-09-05T01:15:24.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.944+0000 7f088ca1c640 1 -- 172.21.15.129:0/835217142 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f087009b270 con 0x7f0888101c50 2024-09-05T01:15:24.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 msgr2=0x7f08740a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:24.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 0x7f08740a51a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0878000c00 tx=0x7f08780314a0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 msgr2=0x7f088819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f087000d770 tx=0x7f087000dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 shutdown_connections 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f08740a2ce0 0x7f08740a51a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0888102610 0x7f08881a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0888101c50 0x7f088819ee60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 --2- 172.21.15.129:0/835217142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0888100a60 0x7f088806ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 >> 172.21.15.129:0/835217142 conn(0x7f08880fc1c0 msgr2=0x7f0888105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 shutdown_connections 2024-09-05T01:15:24.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:24.947+0000 7f088da1e640 1 -- 172.21.15.129:0/835217142 wait complete. 2024-09-05T01:15:24.959 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-05T01:15:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 -- 172.21.15.129:0/2195601983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178102060 msgr2=0x7fd178102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 --2- 172.21.15.129:0/2195601983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178102060 0x7fd178102460 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd168009900 tx=0x7fd16802a410 comp rx=0 tx=0).stop 2024-09-05T01:15:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 -- 172.21.15.129:0/2195601983 shutdown_connections 2024-09-05T01:15:25.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 --2- 172.21.15.129:0/2195601983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178101670 0x7fd178108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 --2- 172.21.15.129:0/2195601983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd178101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 --2- 172.21.15.129:0/2195601983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178102060 0x7fd178102460 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 -- 172.21.15.129:0/2195601983 >> 172.21.15.129:0/2195601983 conn(0x7fd1780fb970 msgr2=0x7fd1780fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 -- 172.21.15.129:0/2195601983 shutdown_connections 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 -- 172.21.15.129:0/2195601983 wait complete. 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.062+0000 7fd17da82640 1 Processor -- start 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 -- start start 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd17819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 0x7fd1781a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:25.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810d7a0 con 0x7fd178101670 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810d620 con 0x7fd178100cb0 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd17810d920 con 0x7fd178102060 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52154/0 (socket says 172.21.15.129:52154) 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 -- 172.21.15.129:0/950667475 learned_addr learned my addr 172.21.15.129:0/950667475 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd1777fe640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd17819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd177fff640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 0x7fd1781a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 -- 172.21.15.129:0/950667475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 msgr2=0x7fd1781a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 0x7fd1781a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 -- 172.21.15.129:0/950667475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 msgr2=0x7fd17819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd17819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 -- 172.21.15.129:0/950667475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1781a1d00 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd177fff640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 0x7fd1781a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd1777fe640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd17819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd16ffff640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b450 tx=0x7fd16000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160015020 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd160011d40 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1781a1ff0 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.063+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16000f340 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.064+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1781a24e0 con 0x7fd178101670 2024-09-05T01:15:25.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.065+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1781a27a0 con 0x7fd178101670 2024-09-05T01:15:25.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.066+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd160018020 con 0x7fd178101670 2024-09-05T01:15:25.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.066+0000 7fd1757fa640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:25.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.066+0000 7fd1777fe640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:25.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.066+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fd1600cef20 con 0x7fd178101670 2024-09-05T01:15:25.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.067+0000 7fd1777fe640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 0x7fd1640a5170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd168000c00 tx=0x7fd168031d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:25.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.070+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd16001f020 con 0x7fd178101670 2024-09-05T01:15:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:24 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-05T01:15:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:24 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-05T01:15:25.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.264+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fd178108bf0 con 0x7fd1640a2cb0 2024-09-05T01:15:25.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.277+0000 7fd1757fa640 1 -- 172.21.15.129:0/950667475 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fd178108bf0 con 0x7fd1640a2cb0 2024-09-05T01:15:25.278 INFO:teuthology.orchestra.run.smithi129.stdout:{} 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 msgr2=0x7fd1640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 0x7fd1640a5170 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd168000c00 tx=0x7fd168031d10 comp rx=0 tx=0).stop 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 msgr2=0x7fd17819a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd16000b450 tx=0x7fd16000b910 comp rx=0 tx=0).stop 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 shutdown_connections 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd178102060 0x7fd1781a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd178101670 0x7fd17819a620 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 --2- 172.21.15.129:0/950667475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd178100cb0 0x7fd17819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:25.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 >> 172.21.15.129:0/950667475 conn(0x7fd1780fb970 msgr2=0x7fd1781074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:25.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 shutdown_connections 2024-09-05T01:15:25.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:25.279+0000 7fd17da82640 1 -- 172.21.15.129:0/950667475 wait complete. 2024-09-05T01:15:25.291 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-05T01:15:25.304 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-05T01:15:25.313 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:15:25.314 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-05T01:15:25.316 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-05T01:15:25.328 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T01:15:25.328 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:15:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:24 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-05T01:15:25.574 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:25.696 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:15:25.269+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:15:25.696 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:15:25.269+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:15:25.697 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:15:25.269+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:15:25.697 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:15:25.269+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:15:25.697 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:15:25.269+0000 7f26ff01a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T01:15:26.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 -- 172.21.15.129:0/179156862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102570 msgr2=0x7fd7141029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:26.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 --2- 172.21.15.129:0/179156862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102570 0x7fd7141029b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc009900 tx=0x7fd6fc02a480 comp rx=0 tx=0).stop 2024-09-05T01:15:26.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 -- 172.21.15.129:0/179156862 shutdown_connections 2024-09-05T01:15:26.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 --2- 172.21.15.129:0/179156862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102ef0 0x7fd71410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 --2- 172.21.15.129:0/179156862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102570 0x7fd7141029b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 --2- 172.21.15.129:0/179156862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd714101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 -- 172.21.15.129:0/179156862 >> 172.21.15.129:0/179156862 conn(0x7fd7140fc7d0 msgr2=0x7fd7140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 -- 172.21.15.129:0/179156862 shutdown_connections 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.072+0000 7fd719941640 1 -- 172.21.15.129:0/179156862 wait complete. 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.073+0000 7fd719941640 1 Processor -- start 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.073+0000 7fd719941640 1 -- start start 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.073+0000 7fd719941640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd7141970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 0x7fd714197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd714110e70 con 0x7fd714102ef0 2024-09-05T01:15:26.093 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd714110cf0 con 0x7fd714100e20 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd714110ff0 con 0x7fd714102570 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd70a7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 0x7fd714197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd712ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd7141970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52212/0 (socket says 172.21.15.129:52212) 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 -- 172.21.15.129:0/1649132244 learned_addr learned my addr 172.21.15.129:0/1649132244 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 -- 172.21.15.129:0/1649132244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 msgr2=0x7fd714197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 0x7fd714197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 -- 172.21.15.129:0/1649132244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 msgr2=0x7fd7141970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd7141970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.094 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 -- 172.21.15.129:0/1649132244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71419c660 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd70a7fc640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 0x7fd714197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd712ffd640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd7141970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd7137fe640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00eaa0 tx=0x7fd70c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c004d20 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd70c005020 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71419c950 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0116b0 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.074+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd71419ce60 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.075+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7141021b0 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.076+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd70c0051d0 con 0x7fd714102ef0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.077+0000 7fd71893f640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 0x7fd7000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.077+0000 7fd712ffd640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 0x7fd7000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:26.095 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.077+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fd70c0cf760 con 0x7fd714102ef0 2024-09-05T01:15:26.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.078+0000 7fd712ffd640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 0x7fd7000a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd704009d60 tx=0x7fd704009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:26.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.082+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd70c016030 con 0x7fd714102ef0 2024-09-05T01:15:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-05T01:15:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-mon[30979]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:25 smithi129 ceph-mon[30979]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:26.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.245+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd71419d120 con 0x7fd7000a2c10 2024-09-05T01:15:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-05T01:15:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:25 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:25 smithi194 ceph-mon[34941]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:25 smithi194 ceph-mon[34941]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:26.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.253+0000 7fd71893f640 1 -- 172.21.15.129:0/1649132244 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340538 (secure 0 0 0) 0x7fd71419d120 con 0x7fd7000a2c10 2024-09-05T01:15:26.258 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:26.260 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:15:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.256+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 msgr2=0x7fd7000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:26.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.256+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 0x7fd7000a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd704009d60 tx=0x7fd704009110 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 msgr2=0x7fd71419c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00eaa0 tx=0x7fd70c00eea0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 shutdown_connections 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd7000a2c10 0x7fd7000a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd714102ef0 0x7fd71419c120 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd714102570 0x7fd714197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 --2- 172.21.15.129:0/1649132244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd714100e20 0x7fd7141970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 >> 172.21.15.129:0/1649132244 conn(0x7fd7140fc7d0 msgr2=0x7fd7140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 shutdown_connections 2024-09-05T01:15:26.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:26.257+0000 7fd719941640 1 -- 172.21.15.129:0/1649132244 wait complete. 2024-09-05T01:15:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/835217142' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-05T01:15:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:25 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:25 smithi183 ceph-mon[34589]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T01:15:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:25 smithi183 ceph-mon[34589]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 767 B/s wr, 7 op/s; 7.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:26.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:26 smithi129 ceph-mon[30979]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T01:15:26.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:26 smithi129 ceph-mon[30979]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:27.002 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-05T01:15:25.782704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657550,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":714,"num_read_kb":477,"num_write":358,"num_write_kb":3062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1244217,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401028,"kb_used_data":14192,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540796,"statfs":{"total":1151940427776,"available":1151529775104,"internally_reserved":0,"allocated":14532608,"data_stored":8124489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":395752763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":869,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":31,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001914"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876259+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:15:18.876259+0000","last_peered":"2024-09-05T01:15:18.876259+0000","last_clean":"2024-09-05T01:15:18.876259+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:15:18.876259+0000","last_undegraded":"2024-09-05T01:15:18.876259+0000","last_fullsized":"2024-09-05T01:15:18.876259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874746+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:15:18.874746+0000","last_peered":"2024-09-05T01:15:18.874746+0000","last_clean":"2024-09-05T01:15:18.874746+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:15:18.874746+0000","last_undegraded":"2024-09-05T01:15:18.874746+0000","last_fullsized":"2024-09-05T01:15:18.874746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874793+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:15:18.874793+0000","last_peered":"2024-09-05T01:15:18.874793+0000","last_clean":"2024-09-05T01:15:18.874793+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:15:18.874793+0000","last_undegraded":"2024-09-05T01:15:18.874793+0000","last_fullsized":"2024-09-05T01:15:18.874793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875030+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:15:18.875030+0000","last_peered":"2024-09-05T01:15:18.875030+0000","last_clean":"2024-09-05T01:15:18.875030+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:15:18.875030+0000","last_undegraded":"2024-09-05T01:15:18.875030+0000","last_fullsized":"2024-09-05T01:15:18.875030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893074+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:15:18.893074+0000","last_peered":"2024-09-05T01:15:18.893074+0000","last_clean":"2024-09-05T01:15:18.893074+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:15:18.893074+0000","last_undegraded":"2024-09-05T01:15:18.893074+0000","last_fullsized":"2024-09-05T01:15:18.893074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894431+0000","last_change":"2024-09-05T01:14:18.913937+0000","last_active":"2024-09-05T01:15:18.894431+0000","last_peered":"2024-09-05T01:15:18.894431+0000","last_clean":"2024-09-05T01:15:18.894431+0000","last_became_active":"2024-09-05T01:14:18.913425+0000","last_became_peered":"2024-09-05T01:14:18.913425+0000","last_unstale":"2024-09-05T01:15:18.894431+0000","last_undegraded":"2024-09-05T01:15:18.894431+0000","last_fullsized":"2024-09-05T01:15:18.894431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:56.174787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878167+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:15:18.878167+0000","last_peered":"2024-09-05T01:15:18.878167+0000","last_clean":"2024-09-05T01:15:18.878167+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:15:18.878167+0000","last_undegraded":"2024-09-05T01:15:18.878167+0000","last_fullsized":"2024-09-05T01:15:18.878167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873242+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:15:18.873242+0000","last_peered":"2024-09-05T01:15:18.873242+0000","last_clean":"2024-09-05T01:15:18.873242+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:15:18.873242+0000","last_undegraded":"2024-09-05T01:15:18.873242+0000","last_fullsized":"2024-09-05T01:15:18.873242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894484+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:15:18.894484+0000","last_peered":"2024-09-05T01:15:18.894484+0000","last_clean":"2024-09-05T01:15:18.894484+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:15:18.894484+0000","last_undegraded":"2024-09-05T01:15:18.894484+0000","last_fullsized":"2024-09-05T01:15:18.894484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878648+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:15:18.878648+0000","last_peered":"2024-09-05T01:15:18.878648+0000","last_clean":"2024-09-05T01:15:18.878648+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:15:18.878648+0000","last_undegraded":"2024-09-05T01:15:18.878648+0000","last_fullsized":"2024-09-05T01:15:18.878648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879364+0000","last_change":"2024-09-05T01:14:18.913351+0000","last_active":"2024-09-05T01:15:18.879364+0000","last_peered":"2024-09-05T01:15:18.879364+0000","last_clean":"2024-09-05T01:15:18.879364+0000","last_became_active":"2024-09-05T01:14:18.912935+0000","last_became_peered":"2024-09-05T01:14:18.912935+0000","last_unstale":"2024-09-05T01:15:18.879364+0000","last_undegraded":"2024-09-05T01:15:18.879364+0000","last_fullsized":"2024-09-05T01:15:18.879364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:32:30.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893914+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:15:18.893914+0000","last_peered":"2024-09-05T01:15:18.893914+0000","last_clean":"2024-09-05T01:15:18.893914+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:15:18.893914+0000","last_undegraded":"2024-09-05T01:15:18.893914+0000","last_fullsized":"2024-09-05T01:15:18.893914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877171+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:15:18.877171+0000","last_peered":"2024-09-05T01:15:18.877171+0000","last_clean":"2024-09-05T01:15:18.877171+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:15:18.877171+0000","last_undegraded":"2024-09-05T01:15:18.877171+0000","last_fullsized":"2024-09-05T01:15:18.877171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886549+0000","last_change":"2024-09-05T01:15:18.886013+0000","last_active":"2024-09-05T01:15:18.886549+0000","last_peered":"2024-09-05T01:15:18.886549+0000","last_clean":"2024-09-05T01:15:18.886549+0000","last_became_active":"2024-09-05T01:15:18.881459+0000","last_became_peered":"2024-09-05T01:15:18.881459+0000","last_unstale":"2024-09-05T01:15:18.886549+0000","last_undegraded":"2024-09-05T01:15:18.886549+0000","last_fullsized":"2024-09-05T01:15:18.886549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:59:40.021197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894054+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:15:18.894054+0000","last_peered":"2024-09-05T01:15:18.894054+0000","last_clean":"2024-09-05T01:15:18.894054+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:15:18.894054+0000","last_undegraded":"2024-09-05T01:15:18.894054+0000","last_fullsized":"2024-09-05T01:15:18.894054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874595+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:15:18.874595+0000","last_peered":"2024-09-05T01:15:18.874595+0000","last_clean":"2024-09-05T01:15:18.874595+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:15:18.874595+0000","last_undegraded":"2024-09-05T01:15:18.874595+0000","last_fullsized":"2024-09-05T01:15:18.874595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874871+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:15:18.874871+0000","last_peered":"2024-09-05T01:15:18.874871+0000","last_clean":"2024-09-05T01:15:18.874871+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:15:18.874871+0000","last_undegraded":"2024-09-05T01:15:18.874871+0000","last_fullsized":"2024-09-05T01:15:18.874871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894639+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:15:18.894639+0000","last_peered":"2024-09-05T01:15:18.894639+0000","last_clean":"2024-09-05T01:15:18.894639+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:15:18.894639+0000","last_undegraded":"2024-09-05T01:15:18.894639+0000","last_fullsized":"2024-09-05T01:15:18.894639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877791+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:15:18.877791+0000","last_peered":"2024-09-05T01:15:18.877791+0000","last_clean":"2024-09-05T01:15:18.877791+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:15:18.877791+0000","last_undegraded":"2024-09-05T01:15:18.877791+0000","last_fullsized":"2024-09-05T01:15:18.877791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874641+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:15:18.874641+0000","last_peered":"2024-09-05T01:15:18.874641+0000","last_clean":"2024-09-05T01:15:18.874641+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:15:18.874641+0000","last_undegraded":"2024-09-05T01:15:18.874641+0000","last_fullsized":"2024-09-05T01:15:18.874641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876013+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:15:18.876013+0000","last_peered":"2024-09-05T01:15:18.876013+0000","last_clean":"2024-09-05T01:15:18.876013+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:15:18.876013+0000","last_undegraded":"2024-09-05T01:15:18.876013+0000","last_fullsized":"2024-09-05T01:15:18.876013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878123+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:15:18.878123+0000","last_peered":"2024-09-05T01:15:18.878123+0000","last_clean":"2024-09-05T01:15:18.878123+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:15:18.878123+0000","last_undegraded":"2024-09-05T01:15:18.878123+0000","last_fullsized":"2024-09-05T01:15:18.878123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873055+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:15:18.873055+0000","last_peered":"2024-09-05T01:15:18.873055+0000","last_clean":"2024-09-05T01:15:18.873055+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:15:18.873055+0000","last_undegraded":"2024-09-05T01:15:18.873055+0000","last_fullsized":"2024-09-05T01:15:18.873055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894188+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:15:18.894188+0000","last_peered":"2024-09-05T01:15:18.894188+0000","last_clean":"2024-09-05T01:15:18.894188+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:15:18.894188+0000","last_undegraded":"2024-09-05T01:15:18.894188+0000","last_fullsized":"2024-09-05T01:15:18.894188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892951+0000","last_change":"2024-09-05T01:14:18.912689+0000","last_active":"2024-09-05T01:15:18.892951+0000","last_peered":"2024-09-05T01:15:18.892951+0000","last_clean":"2024-09-05T01:15:18.892951+0000","last_became_active":"2024-09-05T01:14:18.912358+0000","last_became_peered":"2024-09-05T01:14:18.912358+0000","last_unstale":"2024-09-05T01:15:18.892951+0000","last_undegraded":"2024-09-05T01:15:18.892951+0000","last_fullsized":"2024-09-05T01:15:18.892951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:59.876348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894446+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:15:18.894446+0000","last_peered":"2024-09-05T01:15:18.894446+0000","last_clean":"2024-09-05T01:15:18.894446+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:15:18.894446+0000","last_undegraded":"2024-09-05T01:15:18.894446+0000","last_fullsized":"2024-09-05T01:15:18.894446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875054+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:15:18.875054+0000","last_peered":"2024-09-05T01:15:18.875054+0000","last_clean":"2024-09-05T01:15:18.875054+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:15:18.875054+0000","last_undegraded":"2024-09-05T01:15:18.875054+0000","last_fullsized":"2024-09-05T01:15:18.875054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875149+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:15:18.875149+0000","last_peered":"2024-09-05T01:15:18.875149+0000","last_clean":"2024-09-05T01:15:18.875149+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:15:18.875149+0000","last_undegraded":"2024-09-05T01:15:18.875149+0000","last_fullsized":"2024-09-05T01:15:18.875149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886618+0000","last_change":"2024-09-05T01:15:18.886355+0000","last_active":"2024-09-05T01:15:18.886618+0000","last_peered":"2024-09-05T01:15:18.886618+0000","last_clean":"2024-09-05T01:15:18.886618+0000","last_became_active":"2024-09-05T01:15:18.881103+0000","last_became_peered":"2024-09-05T01:15:18.881103+0000","last_unstale":"2024-09-05T01:15:18.886618+0000","last_undegraded":"2024-09-05T01:15:18.886618+0000","last_fullsized":"2024-09-05T01:15:18.886618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:34:33.131422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874250+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:15:18.874250+0000","last_peered":"2024-09-05T01:15:18.874250+0000","last_clean":"2024-09-05T01:15:18.874250+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:15:18.874250+0000","last_undegraded":"2024-09-05T01:15:18.874250+0000","last_fullsized":"2024-09-05T01:15:18.874250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872626+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:15:18.872626+0000","last_peered":"2024-09-05T01:15:18.872626+0000","last_clean":"2024-09-05T01:15:18.872626+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:15:18.872626+0000","last_undegraded":"2024-09-05T01:15:18.872626+0000","last_fullsized":"2024-09-05T01:15:18.872626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878065+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:15:18.878065+0000","last_peered":"2024-09-05T01:15:18.878065+0000","last_clean":"2024-09-05T01:15:18.878065+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:15:18.878065+0000","last_undegraded":"2024-09-05T01:15:18.878065+0000","last_fullsized":"2024-09-05T01:15:18.878065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878364+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:15:18.878364+0000","last_peered":"2024-09-05T01:15:18.878364+0000","last_clean":"2024-09-05T01:15:18.878364+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:15:18.878364+0000","last_undegraded":"2024-09-05T01:15:18.878364+0000","last_fullsized":"2024-09-05T01:15:18.878364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875341+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:15:18.875341+0000","last_peered":"2024-09-05T01:15:18.875341+0000","last_clean":"2024-09-05T01:15:18.875341+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:15:18.875341+0000","last_undegraded":"2024-09-05T01:15:18.875341+0000","last_fullsized":"2024-09-05T01:15:18.875341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875954+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:15:18.875954+0000","last_peered":"2024-09-05T01:15:18.875954+0000","last_clean":"2024-09-05T01:15:18.875954+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:15:18.875954+0000","last_undegraded":"2024-09-05T01:15:18.875954+0000","last_fullsized":"2024-09-05T01:15:18.875954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878308+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:15:18.878308+0000","last_peered":"2024-09-05T01:15:18.878308+0000","last_clean":"2024-09-05T01:15:18.878308+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:15:18.878308+0000","last_undegraded":"2024-09-05T01:15:18.878308+0000","last_fullsized":"2024-09-05T01:15:18.878308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875250+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:15:18.875250+0000","last_peered":"2024-09-05T01:15:18.875250+0000","last_clean":"2024-09-05T01:15:18.875250+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:15:18.875250+0000","last_undegraded":"2024-09-05T01:15:18.875250+0000","last_fullsized":"2024-09-05T01:15:18.875250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877310+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:15:18.877310+0000","last_peered":"2024-09-05T01:15:18.877310+0000","last_clean":"2024-09-05T01:15:18.877310+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:15:18.877310+0000","last_undegraded":"2024-09-05T01:15:18.877310+0000","last_fullsized":"2024-09-05T01:15:18.877310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872465+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:15:18.872465+0000","last_peered":"2024-09-05T01:15:18.872465+0000","last_clean":"2024-09-05T01:15:18.872465+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:15:18.872465+0000","last_undegraded":"2024-09-05T01:15:18.872465+0000","last_fullsized":"2024-09-05T01:15:18.872465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875893+0000","last_change":"2024-09-05T01:14:18.912776+0000","last_active":"2024-09-05T01:15:18.875893+0000","last_peered":"2024-09-05T01:15:18.875893+0000","last_clean":"2024-09-05T01:15:18.875893+0000","last_became_active":"2024-09-05T01:14:18.912337+0000","last_became_peered":"2024-09-05T01:14:18.912337+0000","last_unstale":"2024-09-05T01:15:18.875893+0000","last_undegraded":"2024-09-05T01:15:18.875893+0000","last_fullsized":"2024-09-05T01:15:18.875893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:37:57.935580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.990665+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:15:21.990665+0000","last_peered":"2024-09-05T01:15:21.990665+0000","last_clean":"2024-09-05T01:15:21.990665+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:15:21.990665+0000","last_undegraded":"2024-09-05T01:15:21.990665+0000","last_fullsized":"2024-09-05T01:15:21.990665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872383+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:15:18.872383+0000","last_peered":"2024-09-05T01:15:18.872383+0000","last_clean":"2024-09-05T01:15:18.872383+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:15:18.872383+0000","last_undegraded":"2024-09-05T01:15:18.872383+0000","last_fullsized":"2024-09-05T01:15:18.872383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872051+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:15:18.872051+0000","last_peered":"2024-09-05T01:15:18.872051+0000","last_clean":"2024-09-05T01:15:18.872051+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:15:18.872051+0000","last_undegraded":"2024-09-05T01:15:18.872051+0000","last_fullsized":"2024-09-05T01:15:18.872051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.898345+0000","last_change":"2024-09-05T01:15:18.898345+0000","last_active":"2024-09-05T01:15:18.898345+0000","last_peered":"2024-09-05T01:15:18.898345+0000","last_clean":"2024-09-05T01:15:18.898345+0000","last_became_active":"2024-09-05T01:15:18.895355+0000","last_became_peered":"2024-09-05T01:15:18.895355+0000","last_unstale":"2024-09-05T01:15:18.898345+0000","last_undegraded":"2024-09-05T01:15:18.898345+0000","last_fullsized":"2024-09-05T01:15:18.898345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:04.042332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879210+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:15:18.879210+0000","last_peered":"2024-09-05T01:15:18.879210+0000","last_clean":"2024-09-05T01:15:18.879210+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:15:18.879210+0000","last_undegraded":"2024-09-05T01:15:18.879210+0000","last_fullsized":"2024-09-05T01:15:18.879210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874602+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:15:18.874602+0000","last_peered":"2024-09-05T01:15:18.874602+0000","last_clean":"2024-09-05T01:15:18.874602+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:15:18.874602+0000","last_undegraded":"2024-09-05T01:15:18.874602+0000","last_fullsized":"2024-09-05T01:15:18.874602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894609+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:15:18.894609+0000","last_peered":"2024-09-05T01:15:18.894609+0000","last_clean":"2024-09-05T01:15:18.894609+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:15:18.894609+0000","last_undegraded":"2024-09-05T01:15:18.894609+0000","last_fullsized":"2024-09-05T01:15:18.894609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877862+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:15:18.877862+0000","last_peered":"2024-09-05T01:15:18.877862+0000","last_clean":"2024-09-05T01:15:18.877862+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:15:18.877862+0000","last_undegraded":"2024-09-05T01:15:18.877862+0000","last_fullsized":"2024-09-05T01:15:18.877862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877578+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:15:18.877578+0000","last_peered":"2024-09-05T01:15:18.877578+0000","last_clean":"2024-09-05T01:15:18.877578+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:15:18.877578+0000","last_undegraded":"2024-09-05T01:15:18.877578+0000","last_fullsized":"2024-09-05T01:15:18.877578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877709+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:15:18.877709+0000","last_peered":"2024-09-05T01:15:18.877709+0000","last_clean":"2024-09-05T01:15:18.877709+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:15:18.877709+0000","last_undegraded":"2024-09-05T01:15:18.877709+0000","last_fullsized":"2024-09-05T01:15:18.877709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876002+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:15:18.876002+0000","last_peered":"2024-09-05T01:15:18.876002+0000","last_clean":"2024-09-05T01:15:18.876002+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:15:18.876002+0000","last_undegraded":"2024-09-05T01:15:18.876002+0000","last_fullsized":"2024-09-05T01:15:18.876002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872919+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:15:18.872919+0000","last_peered":"2024-09-05T01:15:18.872919+0000","last_clean":"2024-09-05T01:15:18.872919+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:15:18.872919+0000","last_undegraded":"2024-09-05T01:15:18.872919+0000","last_fullsized":"2024-09-05T01:15:18.872919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877435+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:15:18.877435+0000","last_peered":"2024-09-05T01:15:18.877435+0000","last_clean":"2024-09-05T01:15:18.877435+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:15:18.877435+0000","last_undegraded":"2024-09-05T01:15:18.877435+0000","last_fullsized":"2024-09-05T01:15:18.877435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879901+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:15:18.879901+0000","last_peered":"2024-09-05T01:15:18.879901+0000","last_clean":"2024-09-05T01:15:18.879901+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:15:18.879901+0000","last_undegraded":"2024-09-05T01:15:18.879901+0000","last_fullsized":"2024-09-05T01:15:18.879901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892762+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:15:18.892762+0000","last_peered":"2024-09-05T01:15:18.892762+0000","last_clean":"2024-09-05T01:15:18.892762+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:15:18.892762+0000","last_undegraded":"2024-09-05T01:15:18.892762+0000","last_fullsized":"2024-09-05T01:15:18.892762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877936+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:15:18.877936+0000","last_peered":"2024-09-05T01:15:18.877936+0000","last_clean":"2024-09-05T01:15:18.877936+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:15:18.877936+0000","last_undegraded":"2024-09-05T01:15:18.877936+0000","last_fullsized":"2024-09-05T01:15:18.877936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879564+0000","last_change":"2024-09-05T01:15:18.879564+0000","last_active":"2024-09-05T01:15:18.879564+0000","last_peered":"2024-09-05T01:15:18.879564+0000","last_clean":"2024-09-05T01:15:18.879564+0000","last_became_active":"2024-09-05T01:15:18.876215+0000","last_became_peered":"2024-09-05T01:15:18.876215+0000","last_unstale":"2024-09-05T01:15:18.879564+0000","last_undegraded":"2024-09-05T01:15:18.879564+0000","last_fullsized":"2024-09-05T01:15:18.879564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:54:11.502597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877679+0000","last_change":"2024-09-05T01:14:18.913738+0000","last_active":"2024-09-05T01:15:18.877679+0000","last_peered":"2024-09-05T01:15:18.877679+0000","last_clean":"2024-09-05T01:15:18.877679+0000","last_became_active":"2024-09-05T01:14:18.913084+0000","last_became_peered":"2024-09-05T01:14:18.913084+0000","last_unstale":"2024-09-05T01:15:18.877679+0000","last_undegraded":"2024-09-05T01:15:18.877679+0000","last_fullsized":"2024-09-05T01:15:18.877679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:43.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879092+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:15:18.879092+0000","last_peered":"2024-09-05T01:15:18.879092+0000","last_clean":"2024-09-05T01:15:18.879092+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:15:18.879092+0000","last_undegraded":"2024-09-05T01:15:18.879092+0000","last_fullsized":"2024-09-05T01:15:18.879092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.880937+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:15:18.880937+0000","last_peered":"2024-09-05T01:15:18.880937+0000","last_clean":"2024-09-05T01:15:18.880937+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:15:18.880937+0000","last_undegraded":"2024-09-05T01:15:18.880937+0000","last_fullsized":"2024-09-05T01:15:18.880937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876614+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:15:18.876614+0000","last_peered":"2024-09-05T01:15:18.876614+0000","last_clean":"2024-09-05T01:15:18.876614+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:15:18.876614+0000","last_undegraded":"2024-09-05T01:15:18.876614+0000","last_fullsized":"2024-09-05T01:15:18.876614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876672+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:15:18.876672+0000","last_peered":"2024-09-05T01:15:18.876672+0000","last_clean":"2024-09-05T01:15:18.876672+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:15:18.876672+0000","last_undegraded":"2024-09-05T01:15:18.876672+0000","last_fullsized":"2024-09-05T01:15:18.876672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877975+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:15:18.877975+0000","last_peered":"2024-09-05T01:15:18.877975+0000","last_clean":"2024-09-05T01:15:18.877975+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:15:18.877975+0000","last_undegraded":"2024-09-05T01:15:18.877975+0000","last_fullsized":"2024-09-05T01:15:18.877975+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872466+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:15:18.872466+0000","last_peered":"2024-09-05T01:15:18.872466+0000","last_clean":"2024-09-05T01:15:18.872466+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:15:18.872466+0000","last_undegraded":"2024-09-05T01:15:18.872466+0000","last_fullsized":"2024-09-05T01:15:18.872466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877006+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:15:18.877006+0000","last_peered":"2024-09-05T01:15:18.877006+0000","last_clean":"2024-09-05T01:15:18.877006+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:15:18.877006+0000","last_undegraded":"2024-09-05T01:15:18.877006+0000","last_fullsized":"2024-09-05T01:15:18.877006+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:19.988205+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:15:19.988205+0000","last_peered":"2024-09-05T01:15:19.988205+0000","last_clean":"2024-09-05T01:15:19.988205+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:15:19.988205+0000","last_undegraded":"2024-09-05T01:15:19.988205+0000","last_fullsized":"2024-09-05T01:15:19.988205+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876080+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:15:18.876080+0000","last_peered":"2024-09-05T01:15:18.876080+0000","last_clean":"2024-09-05T01:15:18.876080+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:15:18.876080+0000","last_undegraded":"2024-09-05T01:15:18.876080+0000","last_fullsized":"2024-09-05T01:15:18.876080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877113+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:15:18.877113+0000","last_peered":"2024-09-05T01:15:18.877113+0000","last_clean":"2024-09-05T01:15:18.877113+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:15:18.877113+0000","last_undegraded":"2024-09-05T01:15:18.877113+0000","last_fullsized":"2024-09-05T01:15:18.877113+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877183+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:15:18.877183+0000","last_peered":"2024-09-05T01:15:18.877183+0000","last_clean":"2024-09-05T01:15:18.877183+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:15:18.877183+0000","last_undegraded":"2024-09-05T01:15:18.877183+0000","last_fullsized":"2024-09-05T01:15:18.877183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877256+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:15:18.877256+0000","last_peered":"2024-09-05T01:15:18.877256+0000","last_clean":"2024-09-05T01:15:18.877256+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:15:18.877256+0000","last_undegraded":"2024-09-05T01:15:18.877256+0000","last_fullsized":"2024-09-05T01:15:18.877256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872199+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:15:18.872199+0000","last_peered":"2024-09-05T01:15:18.872199+0000","last_clean":"2024-09-05T01:15:18.872199+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:15:18.872199+0000","last_undegraded":"2024-09-05T01:15:18.872199+0000","last_fullsized":"2024-09-05T01:15:18.872199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875174+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:15:18.875174+0000","last_peered":"2024-09-05T01:15:18.875174+0000","last_clean":"2024-09-05T01:15:18.875174+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:15:18.875174+0000","last_undegraded":"2024-09-05T01:15:18.875174+0000","last_fullsized":"2024-09-05T01:15:18.875174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879784+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:15:18.879784+0000","last_peered":"2024-09-05T01:15:18.879784+0000","last_clean":"2024-09-05T01:15:18.879784+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:15:18.879784+0000","last_undegraded":"2024-09-05T01:15:18.879784+0000","last_fullsized":"2024-09-05T01:15:18.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877835+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:15:18.877835+0000","last_peered":"2024-09-05T01:15:18.877835+0000","last_clean":"2024-09-05T01:15:18.877835+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:15:18.877835+0000","last_undegraded":"2024-09-05T01:15:18.877835+0000","last_fullsized":"2024-09-05T01:15:18.877835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":287,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.904041+0000","last_change":"2024-09-05T01:15:18.902797+0000","last_active":"2024-09-05T01:15:21.904041+0000","last_peered":"2024-09-05T01:15:21.904041+0000","last_clean":"2024-09-05T01:15:21.904041+0000","last_became_active":"2024-09-05T01:15:18.896064+0000","last_became_peered":"2024-09-05T01:15:18.896064+0000","last_unstale":"2024-09-05T01:15:21.904041+0000","last_undegraded":"2024-09-05T01:15:21.904041+0000","last_fullsized":"2024-09-05T01:15:21.904041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:53.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62325,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":159,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62325,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874983+0000","last_change":"2024-09-05T01:14:18.840367+0000","last_active":"2024-09-05T01:15:18.874983+0000","last_peered":"2024-09-05T01:15:18.874983+0000","last_clean":"2024-09-05T01:15:18.874983+0000","last_became_active":"2024-09-05T01:14:18.839937+0000","last_became_peered":"2024-09-05T01:14:18.839937+0000","last_unstale":"2024-09-05T01:15:18.874983+0000","last_undegraded":"2024-09-05T01:15:18.874983+0000","last_fullsized":"2024-09-05T01:15:18.874983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:49:14.286631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873258+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:15:18.873258+0000","last_peered":"2024-09-05T01:15:18.873258+0000","last_clean":"2024-09-05T01:15:18.873258+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:15:18.873258+0000","last_undegraded":"2024-09-05T01:15:18.873258+0000","last_fullsized":"2024-09-05T01:15:18.873258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879574+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:15:18.879574+0000","last_peered":"2024-09-05T01:15:18.879574+0000","last_clean":"2024-09-05T01:15:18.879574+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:15:18.879574+0000","last_undegraded":"2024-09-05T01:15:18.879574+0000","last_fullsized":"2024-09-05T01:15:18.879574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894131+0000","last_change":"2024-09-05T01:14:18.913011+0000","last_active":"2024-09-05T01:15:18.894131+0000","last_peered":"2024-09-05T01:15:18.894131+0000","last_clean":"2024-09-05T01:15:18.894131+0000","last_became_active":"2024-09-05T01:14:18.912546+0000","last_became_peered":"2024-09-05T01:14:18.912546+0000","last_unstale":"2024-09-05T01:15:18.894131+0000","last_undegraded":"2024-09-05T01:15:18.894131+0000","last_fullsized":"2024-09-05T01:15:18.894131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:11:02.202754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893072+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:15:18.893072+0000","last_peered":"2024-09-05T01:15:18.893072+0000","last_clean":"2024-09-05T01:15:18.893072+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:15:18.893072+0000","last_undegraded":"2024-09-05T01:15:18.893072+0000","last_fullsized":"2024-09-05T01:15:18.893072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877804+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:15:18.877804+0000","last_peered":"2024-09-05T01:15:18.877804+0000","last_clean":"2024-09-05T01:15:18.877804+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:15:18.877804+0000","last_undegraded":"2024-09-05T01:15:18.877804+0000","last_fullsized":"2024-09-05T01:15:18.877804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875496+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:15:18.875496+0000","last_peered":"2024-09-05T01:15:18.875496+0000","last_clean":"2024-09-05T01:15:18.875496+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:15:18.875496+0000","last_undegraded":"2024-09-05T01:15:18.875496+0000","last_fullsized":"2024-09-05T01:15:18.875496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873793+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:15:18.873793+0000","last_peered":"2024-09-05T01:15:18.873793+0000","last_clean":"2024-09-05T01:15:18.873793+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:15:18.873793+0000","last_undegraded":"2024-09-05T01:15:18.873793+0000","last_fullsized":"2024-09-05T01:15:18.873793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:20.892189+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:15:20.892189+0000","last_peered":"2024-09-05T01:15:20.892189+0000","last_clean":"2024-09-05T01:15:20.892189+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:15:20.892189+0000","last_undegraded":"2024-09-05T01:15:20.892189+0000","last_fullsized":"2024-09-05T01:15:20.892189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874862+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:15:18.874862+0000","last_peered":"2024-09-05T01:15:18.874862+0000","last_clean":"2024-09-05T01:15:18.874862+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:15:18.874862+0000","last_undegraded":"2024-09-05T01:15:18.874862+0000","last_fullsized":"2024-09-05T01:15:18.874862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892843+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:15:18.892843+0000","last_peered":"2024-09-05T01:15:18.892843+0000","last_clean":"2024-09-05T01:15:18.892843+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:15:18.892843+0000","last_undegraded":"2024-09-05T01:15:18.892843+0000","last_fullsized":"2024-09-05T01:15:18.892843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875596+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:15:18.875596+0000","last_peered":"2024-09-05T01:15:18.875596+0000","last_clean":"2024-09-05T01:15:18.875596+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:15:18.875596+0000","last_undegraded":"2024-09-05T01:15:18.875596+0000","last_fullsized":"2024-09-05T01:15:18.875596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876372+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:15:18.876372+0000","last_peered":"2024-09-05T01:15:18.876372+0000","last_clean":"2024-09-05T01:15:18.876372+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:15:18.876372+0000","last_undegraded":"2024-09-05T01:15:18.876372+0000","last_fullsized":"2024-09-05T01:15:18.876372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879805+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:15:18.879805+0000","last_peered":"2024-09-05T01:15:18.879805+0000","last_clean":"2024-09-05T01:15:18.879805+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:15:18.879805+0000","last_undegraded":"2024-09-05T01:15:18.879805+0000","last_fullsized":"2024-09-05T01:15:18.879805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874509+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:15:18.874509+0000","last_peered":"2024-09-05T01:15:18.874509+0000","last_clean":"2024-09-05T01:15:18.874509+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:15:18.874509+0000","last_undegraded":"2024-09-05T01:15:18.874509+0000","last_fullsized":"2024-09-05T01:15:18.874509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872057+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:15:18.872057+0000","last_peered":"2024-09-05T01:15:18.872057+0000","last_clean":"2024-09-05T01:15:18.872057+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:15:18.872057+0000","last_undegraded":"2024-09-05T01:15:18.872057+0000","last_fullsized":"2024-09-05T01:15:18.872057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875691+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:15:18.875691+0000","last_peered":"2024-09-05T01:15:18.875691+0000","last_clean":"2024-09-05T01:15:18.875691+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:15:18.875691+0000","last_undegraded":"2024-09-05T01:15:18.875691+0000","last_fullsized":"2024-09-05T01:15:18.875691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875643+0000","last_change":"2024-09-05T01:14:18.912517+0000","last_active":"2024-09-05T01:15:18.875643+0000","last_peered":"2024-09-05T01:15:18.875643+0000","last_clean":"2024-09-05T01:15:18.875643+0000","last_became_active":"2024-09-05T01:14:18.912074+0000","last_became_peered":"2024-09-05T01:14:18.912074+0000","last_unstale":"2024-09-05T01:15:18.875643+0000","last_undegraded":"2024-09-05T01:15:18.875643+0000","last_fullsized":"2024-09-05T01:15:18.875643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:16.172775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874931+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:15:18.874931+0000","last_peered":"2024-09-05T01:15:18.874931+0000","last_clean":"2024-09-05T01:15:18.874931+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:15:18.874931+0000","last_undegraded":"2024-09-05T01:15:18.874931+0000","last_fullsized":"2024-09-05T01:15:18.874931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:23.898003+0000","last_change":"2024-09-05T01:15:18.891283+0000","last_active":"2024-09-05T01:15:23.898003+0000","last_peered":"2024-09-05T01:15:23.898003+0000","last_clean":"2024-09-05T01:15:23.898003+0000","last_became_active":"2024-09-05T01:15:18.882289+0000","last_became_peered":"2024-09-05T01:15:18.882289+0000","last_unstale":"2024-09-05T01:15:23.898003+0000","last_undegraded":"2024-09-05T01:15:23.898003+0000","last_fullsized":"2024-09-05T01:15:23.898003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:45:12.506629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878425+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:15:18.878425+0000","last_peered":"2024-09-05T01:15:18.878425+0000","last_clean":"2024-09-05T01:15:18.878425+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:15:18.878425+0000","last_undegraded":"2024-09-05T01:15:18.878425+0000","last_fullsized":"2024-09-05T01:15:18.878425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875180+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:15:18.875180+0000","last_peered":"2024-09-05T01:15:18.875180+0000","last_clean":"2024-09-05T01:15:18.875180+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:15:18.875180+0000","last_undegraded":"2024-09-05T01:15:18.875180+0000","last_fullsized":"2024-09-05T01:15:18.875180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877663+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:15:18.877663+0000","last_peered":"2024-09-05T01:15:18.877663+0000","last_clean":"2024-09-05T01:15:18.877663+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:15:18.877663+0000","last_undegraded":"2024-09-05T01:15:18.877663+0000","last_fullsized":"2024-09-05T01:15:18.877663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875455+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:15:18.875455+0000","last_peered":"2024-09-05T01:15:18.875455+0000","last_clean":"2024-09-05T01:15:18.875455+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:15:18.875455+0000","last_undegraded":"2024-09-05T01:15:18.875455+0000","last_fullsized":"2024-09-05T01:15:18.875455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874866+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:15:18.874866+0000","last_peered":"2024-09-05T01:15:18.874866+0000","last_clean":"2024-09-05T01:15:18.874866+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:15:18.874866+0000","last_undegraded":"2024-09-05T01:15:18.874866+0000","last_fullsized":"2024-09-05T01:15:18.874866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892838+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:15:18.892838+0000","last_peered":"2024-09-05T01:15:18.892838+0000","last_clean":"2024-09-05T01:15:18.892838+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:15:18.892838+0000","last_undegraded":"2024-09-05T01:15:18.892838+0000","last_fullsized":"2024-09-05T01:15:18.892838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872553+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:15:18.872553+0000","last_peered":"2024-09-05T01:15:18.872553+0000","last_clean":"2024-09-05T01:15:18.872553+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:15:18.872553+0000","last_undegraded":"2024-09-05T01:15:18.872553+0000","last_fullsized":"2024-09-05T01:15:18.872553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878714+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:15:18.878714+0000","last_peered":"2024-09-05T01:15:18.878714+0000","last_clean":"2024-09-05T01:15:18.878714+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:15:18.878714+0000","last_undegraded":"2024-09-05T01:15:18.878714+0000","last_fullsized":"2024-09-05T01:15:18.878714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876503+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:15:18.876503+0000","last_peered":"2024-09-05T01:15:18.876503+0000","last_clean":"2024-09-05T01:15:18.876503+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:15:18.876503+0000","last_undegraded":"2024-09-05T01:15:18.876503+0000","last_fullsized":"2024-09-05T01:15:18.876503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875119+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:15:18.875119+0000","last_peered":"2024-09-05T01:15:18.875119+0000","last_clean":"2024-09-05T01:15:18.875119+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:15:18.875119+0000","last_undegraded":"2024-09-05T01:15:18.875119+0000","last_fullsized":"2024-09-05T01:15:18.875119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875850+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:15:18.875850+0000","last_peered":"2024-09-05T01:15:18.875850+0000","last_clean":"2024-09-05T01:15:18.875850+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:15:18.875850+0000","last_undegraded":"2024-09-05T01:15:18.875850+0000","last_fullsized":"2024-09-05T01:15:18.875850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874525+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:15:18.874525+0000","last_peered":"2024-09-05T01:15:18.874525+0000","last_clean":"2024-09-05T01:15:18.874525+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:15:18.874525+0000","last_undegraded":"2024-09-05T01:15:18.874525+0000","last_fullsized":"2024-09-05T01:15:18.874525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873202+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:15:18.873202+0000","last_peered":"2024-09-05T01:15:18.873202+0000","last_clean":"2024-09-05T01:15:18.873202+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:15:18.873202+0000","last_undegraded":"2024-09-05T01:15:18.873202+0000","last_fullsized":"2024-09-05T01:15:18.873202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872496+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:15:18.872496+0000","last_peered":"2024-09-05T01:15:18.872496+0000","last_clean":"2024-09-05T01:15:18.872496+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:15:18.872496+0000","last_undegraded":"2024-09-05T01:15:18.872496+0000","last_fullsized":"2024-09-05T01:15:18.872496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877437+0000","last_change":"2024-09-05T01:14:18.840355+0000","last_active":"2024-09-05T01:15:18.877437+0000","last_peered":"2024-09-05T01:15:18.877437+0000","last_clean":"2024-09-05T01:15:18.877437+0000","last_became_active":"2024-09-05T01:14:18.839887+0000","last_became_peered":"2024-09-05T01:14:18.839887+0000","last_unstale":"2024-09-05T01:15:18.877437+0000","last_undegraded":"2024-09-05T01:15:18.877437+0000","last_fullsized":"2024-09-05T01:15:18.877437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:49.152575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875539+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:15:18.875539+0000","last_peered":"2024-09-05T01:15:18.875539+0000","last_clean":"2024-09-05T01:15:18.875539+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:15:18.875539+0000","last_undegraded":"2024-09-05T01:15:18.875539+0000","last_fullsized":"2024-09-05T01:15:18.875539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876230+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:15:18.876230+0000","last_peered":"2024-09-05T01:15:18.876230+0000","last_clean":"2024-09-05T01:15:18.876230+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:15:18.876230+0000","last_undegraded":"2024-09-05T01:15:18.876230+0000","last_fullsized":"2024-09-05T01:15:18.876230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879606+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:15:18.879606+0000","last_peered":"2024-09-05T01:15:18.879606+0000","last_clean":"2024-09-05T01:15:18.879606+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:15:18.879606+0000","last_undegraded":"2024-09-05T01:15:18.879606+0000","last_fullsized":"2024-09-05T01:15:18.879606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872781+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:15:18.872781+0000","last_peered":"2024-09-05T01:15:18.872781+0000","last_clean":"2024-09-05T01:15:18.872781+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:15:18.872781+0000","last_undegraded":"2024-09-05T01:15:18.872781+0000","last_fullsized":"2024-09-05T01:15:18.872781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892640+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:15:18.892640+0000","last_peered":"2024-09-05T01:15:18.892640+0000","last_clean":"2024-09-05T01:15:18.892640+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:15:18.892640+0000","last_undegraded":"2024-09-05T01:15:18.892640+0000","last_fullsized":"2024-09-05T01:15:18.892640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872317+0000","last_change":"2024-09-05T01:14:18.836841+0000","last_active":"2024-09-05T01:15:18.872317+0000","last_peered":"2024-09-05T01:15:18.872317+0000","last_clean":"2024-09-05T01:15:18.872317+0000","last_became_active":"2024-09-05T01:14:18.836464+0000","last_became_peered":"2024-09-05T01:14:18.836464+0000","last_unstale":"2024-09-05T01:15:18.872317+0000","last_undegraded":"2024-09-05T01:15:18.872317+0000","last_fullsized":"2024-09-05T01:15:18.872317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:21:02.858269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894772+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:15:18.894772+0000","last_peered":"2024-09-05T01:15:18.894772+0000","last_clean":"2024-09-05T01:15:18.894772+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:15:18.894772+0000","last_undegraded":"2024-09-05T01:15:18.894772+0000","last_fullsized":"2024-09-05T01:15:18.894772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877636+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:15:18.877636+0000","last_peered":"2024-09-05T01:15:18.877636+0000","last_clean":"2024-09-05T01:15:18.877636+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:15:18.877636+0000","last_undegraded":"2024-09-05T01:15:18.877636+0000","last_fullsized":"2024-09-05T01:15:18.877636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875974+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:15:18.875974+0000","last_peered":"2024-09-05T01:15:18.875974+0000","last_clean":"2024-09-05T01:15:18.875974+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:15:18.875974+0000","last_undegraded":"2024-09-05T01:15:18.875974+0000","last_fullsized":"2024-09-05T01:15:18.875974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873830+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:15:18.873830+0000","last_peered":"2024-09-05T01:15:18.873830+0000","last_clean":"2024-09-05T01:15:18.873830+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:15:18.873830+0000","last_undegraded":"2024-09-05T01:15:18.873830+0000","last_fullsized":"2024-09-05T01:15:18.873830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874356+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:15:18.874356+0000","last_peered":"2024-09-05T01:15:18.874356+0000","last_clean":"2024-09-05T01:15:18.874356+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:15:18.874356+0000","last_undegraded":"2024-09-05T01:15:18.874356+0000","last_fullsized":"2024-09-05T01:15:18.874356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877120+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:15:18.877120+0000","last_peered":"2024-09-05T01:15:18.877120+0000","last_clean":"2024-09-05T01:15:18.877120+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:15:18.877120+0000","last_undegraded":"2024-09-05T01:15:18.877120+0000","last_fullsized":"2024-09-05T01:15:18.877120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875187+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:15:18.875187+0000","last_peered":"2024-09-05T01:15:18.875187+0000","last_clean":"2024-09-05T01:15:18.875187+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:15:18.875187+0000","last_undegraded":"2024-09-05T01:15:18.875187+0000","last_fullsized":"2024-09-05T01:15:18.875187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879887+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:15:18.879887+0000","last_peered":"2024-09-05T01:15:18.879887+0000","last_clean":"2024-09-05T01:15:18.879887+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:15:18.879887+0000","last_undegraded":"2024-09-05T01:15:18.879887+0000","last_fullsized":"2024-09-05T01:15:18.879887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894875+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:15:18.894875+0000","last_peered":"2024-09-05T01:15:18.894875+0000","last_clean":"2024-09-05T01:15:18.894875+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:15:18.894875+0000","last_undegraded":"2024-09-05T01:15:18.894875+0000","last_fullsized":"2024-09-05T01:15:18.894875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892807+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:15:18.892807+0000","last_peered":"2024-09-05T01:15:18.892807+0000","last_clean":"2024-09-05T01:15:18.892807+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:15:18.892807+0000","last_undegraded":"2024-09-05T01:15:18.892807+0000","last_fullsized":"2024-09-05T01:15:18.892807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879435+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:15:18.879435+0000","last_peered":"2024-09-05T01:15:18.879435+0000","last_clean":"2024-09-05T01:15:18.879435+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:15:18.879435+0000","last_undegraded":"2024-09-05T01:15:18.879435+0000","last_fullsized":"2024-09-05T01:15:18.879435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66948,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":588,"num_read_kb":368,"num_write":175,"num_write_kb":122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":516334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1724416,"data_stored":1164292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":519031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201336,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":514855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1642496,"data_stored":1110907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593795,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":510111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1105339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":577086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215185,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1040384,"data_stored":510305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1048576,"data_stored":511905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":572825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":511499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:15:27.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:15:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:26 smithi194 ceph-mon[34941]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T01:15:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:26 smithi194 ceph-mon[34941]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:27.258 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:26 smithi183 ceph-mon[34589]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-05T01:15:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:26 smithi183 ceph-mon[34589]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 -- 172.21.15.129:0/1359236549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0102790 msgr2=0x7f73c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 --2- 172.21.15.129:0/1359236549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0102790 0x7f73c01067e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009900 tx=0x7f73b002a410 comp rx=0 tx=0).stop 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 -- 172.21.15.129:0/1359236549 shutdown_connections 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 --2- 172.21.15.129:0/1359236549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0106f10 0x7f73c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 --2- 172.21.15.129:0/1359236549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0102790 0x7f73c01067e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 --2- 172.21.15.129:0/1359236549 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0101de0 0x7f73c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 -- 172.21.15.129:0/1359236549 >> 172.21.15.129:0/1359236549 conn(0x7f73c00fd8c0 msgr2=0x7f73c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 -- 172.21.15.129:0/1359236549 shutdown_connections 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.730+0000 7f73c72b8640 1 -- 172.21.15.129:0/1359236549 wait complete. 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 Processor -- start 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 -- start start 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0101de0 0x7f73c01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0102790 0x7f73c0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0111e20 con 0x7f73c0106f10 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0111ca0 con 0x7f73c0101de0 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c72b8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f73c0111fa0 con 0x7f73c0102790 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c5855640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c5855640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52224/0 (socket says 172.21.15.129:52224) 2024-09-05T01:15:27.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c5855640 1 -- 172.21.15.129:0/2877501328 learned_addr learned my addr 172.21.15.129:0/2877501328 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c4853640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0102790 0x7f73c0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.731+0000 7f73c5054640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0101de0 0x7f73c01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 -- 172.21.15.129:0/2877501328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0102790 msgr2=0x7f73c0198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0102790 0x7f73c0198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 -- 172.21.15.129:0/2877501328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0101de0 msgr2=0x7f73c01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0101de0 0x7f73c01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 -- 172.21.15.129:0/2877501328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c019d370 con 0x7f73c0106f10 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c5855640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f73b800cce0 tx=0x7f73b800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b800f450 con 0x7f73c0106f10 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f73b800faa0 con 0x7f73c0106f10 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73b8018b70 con 0x7f73c0106f10 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c019d660 con 0x7f73c0106f10 2024-09-05T01:15:27.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.732+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c01a4f40 con 0x7f73c0106f10 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.733+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c01021c0 con 0x7f73c0106f10 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.734+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f73b8018d10 con 0x7f73c0106f10 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.735+0000 7f73b67fc640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 0x7f73ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.735+0000 7f73c5054640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 0x7f73ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.735+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f73b80cf860 con 0x7f73c0106f10 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.735+0000 7f73c5054640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 0x7f73ac0a5150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005fd0 tx=0x7f73bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:27.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.739+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73b8014030 con 0x7f73c0106f10 2024-09-05T01:15:27.904 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.904+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73c010a1c0 con 0x7f73ac0a2c90 2024-09-05T01:15:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.913+0000 7f73b67fc640 1 -- 172.21.15.129:0/2877501328 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340516 (secure 0 0 0) 0x7f73bc003890 con 0x7f73ac0a2c90 2024-09-05T01:15:27.916 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:27.917 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 msgr2=0x7f73ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 0x7f73ac0a5150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f73bc005fd0 tx=0x7f73bc00b390 comp rx=0 tx=0).stop 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 msgr2=0x7f73c019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f73b800cce0 tx=0x7f73b800a620 comp rx=0 tx=0).stop 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 shutdown_connections 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f73ac0a2c90 0x7f73ac0a5150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73c0106f10 0x7f73c019cc90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f73c0102790 0x7f73c0198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 --2- 172.21.15.129:0/2877501328 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c0101de0 0x7f73c01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:27.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 >> 172.21.15.129:0/2877501328 conn(0x7f73c00fd8c0 msgr2=0x7f73c00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:27.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 shutdown_connections 2024-09-05T01:15:27.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:27.916+0000 7f73c72b8640 1 -- 172.21.15.129:0/2877501328 wait complete. 2024-09-05T01:15:28.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:27 smithi129 ceph-mon[30979]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 206 B/s wr, 4 op/s; 6.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:28.222 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:27 smithi129 ceph-mon[30979]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:27 smithi194 ceph-mon[34941]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 206 B/s wr, 4 op/s; 6.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:27 smithi194 ceph-mon[34941]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:28.311 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:27 smithi183 ceph-mon[34589]: pgmap v355: 129 pgs: 129 active+clean; 643 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 206 B/s wr, 4 op/s; 6.3 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:28.312 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:27 smithi183 ceph-mon[34589]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:28.613 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-05T01:15:27.783121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658029,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":720,"num_read_kb":478,"num_write":359,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1244217,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401008,"kb_used_data":14196,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540816,"statfs":{"total":1151940427776,"available":1151529795584,"internally_reserved":0,"allocated":14536704,"data_stored":8133632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":395752763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":958,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":22,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.913416"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876259+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:15:18.876259+0000","last_peered":"2024-09-05T01:15:18.876259+0000","last_clean":"2024-09-05T01:15:18.876259+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:15:18.876259+0000","last_undegraded":"2024-09-05T01:15:18.876259+0000","last_fullsized":"2024-09-05T01:15:18.876259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874746+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:15:18.874746+0000","last_peered":"2024-09-05T01:15:18.874746+0000","last_clean":"2024-09-05T01:15:18.874746+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:15:18.874746+0000","last_undegraded":"2024-09-05T01:15:18.874746+0000","last_fullsized":"2024-09-05T01:15:18.874746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874793+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:15:18.874793+0000","last_peered":"2024-09-05T01:15:18.874793+0000","last_clean":"2024-09-05T01:15:18.874793+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:15:18.874793+0000","last_undegraded":"2024-09-05T01:15:18.874793+0000","last_fullsized":"2024-09-05T01:15:18.874793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875030+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:15:18.875030+0000","last_peered":"2024-09-05T01:15:18.875030+0000","last_clean":"2024-09-05T01:15:18.875030+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:15:18.875030+0000","last_undegraded":"2024-09-05T01:15:18.875030+0000","last_fullsized":"2024-09-05T01:15:18.875030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893074+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:15:18.893074+0000","last_peered":"2024-09-05T01:15:18.893074+0000","last_clean":"2024-09-05T01:15:18.893074+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:15:18.893074+0000","last_undegraded":"2024-09-05T01:15:18.893074+0000","last_fullsized":"2024-09-05T01:15:18.893074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894431+0000","last_change":"2024-09-05T01:14:18.913937+0000","last_active":"2024-09-05T01:15:18.894431+0000","last_peered":"2024-09-05T01:15:18.894431+0000","last_clean":"2024-09-05T01:15:18.894431+0000","last_became_active":"2024-09-05T01:14:18.913425+0000","last_became_peered":"2024-09-05T01:14:18.913425+0000","last_unstale":"2024-09-05T01:15:18.894431+0000","last_undegraded":"2024-09-05T01:15:18.894431+0000","last_fullsized":"2024-09-05T01:15:18.894431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:56.174787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878167+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:15:18.878167+0000","last_peered":"2024-09-05T01:15:18.878167+0000","last_clean":"2024-09-05T01:15:18.878167+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:15:18.878167+0000","last_undegraded":"2024-09-05T01:15:18.878167+0000","last_fullsized":"2024-09-05T01:15:18.878167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873242+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:15:18.873242+0000","last_peered":"2024-09-05T01:15:18.873242+0000","last_clean":"2024-09-05T01:15:18.873242+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:15:18.873242+0000","last_undegraded":"2024-09-05T01:15:18.873242+0000","last_fullsized":"2024-09-05T01:15:18.873242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894484+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:15:18.894484+0000","last_peered":"2024-09-05T01:15:18.894484+0000","last_clean":"2024-09-05T01:15:18.894484+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:15:18.894484+0000","last_undegraded":"2024-09-05T01:15:18.894484+0000","last_fullsized":"2024-09-05T01:15:18.894484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878648+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:15:18.878648+0000","last_peered":"2024-09-05T01:15:18.878648+0000","last_clean":"2024-09-05T01:15:18.878648+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:15:18.878648+0000","last_undegraded":"2024-09-05T01:15:18.878648+0000","last_fullsized":"2024-09-05T01:15:18.878648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879364+0000","last_change":"2024-09-05T01:14:18.913351+0000","last_active":"2024-09-05T01:15:18.879364+0000","last_peered":"2024-09-05T01:15:18.879364+0000","last_clean":"2024-09-05T01:15:18.879364+0000","last_became_active":"2024-09-05T01:14:18.912935+0000","last_became_peered":"2024-09-05T01:14:18.912935+0000","last_unstale":"2024-09-05T01:15:18.879364+0000","last_undegraded":"2024-09-05T01:15:18.879364+0000","last_fullsized":"2024-09-05T01:15:18.879364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:32:30.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893914+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:15:18.893914+0000","last_peered":"2024-09-05T01:15:18.893914+0000","last_clean":"2024-09-05T01:15:18.893914+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:15:18.893914+0000","last_undegraded":"2024-09-05T01:15:18.893914+0000","last_fullsized":"2024-09-05T01:15:18.893914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877171+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:15:18.877171+0000","last_peered":"2024-09-05T01:15:18.877171+0000","last_clean":"2024-09-05T01:15:18.877171+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:15:18.877171+0000","last_undegraded":"2024-09-05T01:15:18.877171+0000","last_fullsized":"2024-09-05T01:15:18.877171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886549+0000","last_change":"2024-09-05T01:15:18.886013+0000","last_active":"2024-09-05T01:15:18.886549+0000","last_peered":"2024-09-05T01:15:18.886549+0000","last_clean":"2024-09-05T01:15:18.886549+0000","last_became_active":"2024-09-05T01:15:18.881459+0000","last_became_peered":"2024-09-05T01:15:18.881459+0000","last_unstale":"2024-09-05T01:15:18.886549+0000","last_undegraded":"2024-09-05T01:15:18.886549+0000","last_fullsized":"2024-09-05T01:15:18.886549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:59:40.021197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894054+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:15:18.894054+0000","last_peered":"2024-09-05T01:15:18.894054+0000","last_clean":"2024-09-05T01:15:18.894054+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:15:18.894054+0000","last_undegraded":"2024-09-05T01:15:18.894054+0000","last_fullsized":"2024-09-05T01:15:18.894054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874595+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:15:18.874595+0000","last_peered":"2024-09-05T01:15:18.874595+0000","last_clean":"2024-09-05T01:15:18.874595+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:15:18.874595+0000","last_undegraded":"2024-09-05T01:15:18.874595+0000","last_fullsized":"2024-09-05T01:15:18.874595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874871+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:15:18.874871+0000","last_peered":"2024-09-05T01:15:18.874871+0000","last_clean":"2024-09-05T01:15:18.874871+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:15:18.874871+0000","last_undegraded":"2024-09-05T01:15:18.874871+0000","last_fullsized":"2024-09-05T01:15:18.874871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894639+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:15:18.894639+0000","last_peered":"2024-09-05T01:15:18.894639+0000","last_clean":"2024-09-05T01:15:18.894639+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:15:18.894639+0000","last_undegraded":"2024-09-05T01:15:18.894639+0000","last_fullsized":"2024-09-05T01:15:18.894639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877791+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:15:18.877791+0000","last_peered":"2024-09-05T01:15:18.877791+0000","last_clean":"2024-09-05T01:15:18.877791+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:15:18.877791+0000","last_undegraded":"2024-09-05T01:15:18.877791+0000","last_fullsized":"2024-09-05T01:15:18.877791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874641+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:15:18.874641+0000","last_peered":"2024-09-05T01:15:18.874641+0000","last_clean":"2024-09-05T01:15:18.874641+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:15:18.874641+0000","last_undegraded":"2024-09-05T01:15:18.874641+0000","last_fullsized":"2024-09-05T01:15:18.874641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876013+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:15:18.876013+0000","last_peered":"2024-09-05T01:15:18.876013+0000","last_clean":"2024-09-05T01:15:18.876013+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:15:18.876013+0000","last_undegraded":"2024-09-05T01:15:18.876013+0000","last_fullsized":"2024-09-05T01:15:18.876013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878123+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:15:18.878123+0000","last_peered":"2024-09-05T01:15:18.878123+0000","last_clean":"2024-09-05T01:15:18.878123+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:15:18.878123+0000","last_undegraded":"2024-09-05T01:15:18.878123+0000","last_fullsized":"2024-09-05T01:15:18.878123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873055+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:15:18.873055+0000","last_peered":"2024-09-05T01:15:18.873055+0000","last_clean":"2024-09-05T01:15:18.873055+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:15:18.873055+0000","last_undegraded":"2024-09-05T01:15:18.873055+0000","last_fullsized":"2024-09-05T01:15:18.873055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894188+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:15:18.894188+0000","last_peered":"2024-09-05T01:15:18.894188+0000","last_clean":"2024-09-05T01:15:18.894188+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:15:18.894188+0000","last_undegraded":"2024-09-05T01:15:18.894188+0000","last_fullsized":"2024-09-05T01:15:18.894188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892951+0000","last_change":"2024-09-05T01:14:18.912689+0000","last_active":"2024-09-05T01:15:18.892951+0000","last_peered":"2024-09-05T01:15:18.892951+0000","last_clean":"2024-09-05T01:15:18.892951+0000","last_became_active":"2024-09-05T01:14:18.912358+0000","last_became_peered":"2024-09-05T01:14:18.912358+0000","last_unstale":"2024-09-05T01:15:18.892951+0000","last_undegraded":"2024-09-05T01:15:18.892951+0000","last_fullsized":"2024-09-05T01:15:18.892951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:59.876348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894446+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:15:18.894446+0000","last_peered":"2024-09-05T01:15:18.894446+0000","last_clean":"2024-09-05T01:15:18.894446+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:15:18.894446+0000","last_undegraded":"2024-09-05T01:15:18.894446+0000","last_fullsized":"2024-09-05T01:15:18.894446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875054+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:15:18.875054+0000","last_peered":"2024-09-05T01:15:18.875054+0000","last_clean":"2024-09-05T01:15:18.875054+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:15:18.875054+0000","last_undegraded":"2024-09-05T01:15:18.875054+0000","last_fullsized":"2024-09-05T01:15:18.875054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875149+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:15:18.875149+0000","last_peered":"2024-09-05T01:15:18.875149+0000","last_clean":"2024-09-05T01:15:18.875149+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:15:18.875149+0000","last_undegraded":"2024-09-05T01:15:18.875149+0000","last_fullsized":"2024-09-05T01:15:18.875149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886618+0000","last_change":"2024-09-05T01:15:18.886355+0000","last_active":"2024-09-05T01:15:18.886618+0000","last_peered":"2024-09-05T01:15:18.886618+0000","last_clean":"2024-09-05T01:15:18.886618+0000","last_became_active":"2024-09-05T01:15:18.881103+0000","last_became_peered":"2024-09-05T01:15:18.881103+0000","last_unstale":"2024-09-05T01:15:18.886618+0000","last_undegraded":"2024-09-05T01:15:18.886618+0000","last_fullsized":"2024-09-05T01:15:18.886618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:34:33.131422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874250+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:15:18.874250+0000","last_peered":"2024-09-05T01:15:18.874250+0000","last_clean":"2024-09-05T01:15:18.874250+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:15:18.874250+0000","last_undegraded":"2024-09-05T01:15:18.874250+0000","last_fullsized":"2024-09-05T01:15:18.874250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872626+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:15:18.872626+0000","last_peered":"2024-09-05T01:15:18.872626+0000","last_clean":"2024-09-05T01:15:18.872626+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:15:18.872626+0000","last_undegraded":"2024-09-05T01:15:18.872626+0000","last_fullsized":"2024-09-05T01:15:18.872626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878065+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:15:18.878065+0000","last_peered":"2024-09-05T01:15:18.878065+0000","last_clean":"2024-09-05T01:15:18.878065+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:15:18.878065+0000","last_undegraded":"2024-09-05T01:15:18.878065+0000","last_fullsized":"2024-09-05T01:15:18.878065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878364+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:15:18.878364+0000","last_peered":"2024-09-05T01:15:18.878364+0000","last_clean":"2024-09-05T01:15:18.878364+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:15:18.878364+0000","last_undegraded":"2024-09-05T01:15:18.878364+0000","last_fullsized":"2024-09-05T01:15:18.878364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875341+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:15:18.875341+0000","last_peered":"2024-09-05T01:15:18.875341+0000","last_clean":"2024-09-05T01:15:18.875341+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:15:18.875341+0000","last_undegraded":"2024-09-05T01:15:18.875341+0000","last_fullsized":"2024-09-05T01:15:18.875341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875954+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:15:18.875954+0000","last_peered":"2024-09-05T01:15:18.875954+0000","last_clean":"2024-09-05T01:15:18.875954+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:15:18.875954+0000","last_undegraded":"2024-09-05T01:15:18.875954+0000","last_fullsized":"2024-09-05T01:15:18.875954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878308+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:15:18.878308+0000","last_peered":"2024-09-05T01:15:18.878308+0000","last_clean":"2024-09-05T01:15:18.878308+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:15:18.878308+0000","last_undegraded":"2024-09-05T01:15:18.878308+0000","last_fullsized":"2024-09-05T01:15:18.878308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875250+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:15:18.875250+0000","last_peered":"2024-09-05T01:15:18.875250+0000","last_clean":"2024-09-05T01:15:18.875250+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:15:18.875250+0000","last_undegraded":"2024-09-05T01:15:18.875250+0000","last_fullsized":"2024-09-05T01:15:18.875250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877310+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:15:18.877310+0000","last_peered":"2024-09-05T01:15:18.877310+0000","last_clean":"2024-09-05T01:15:18.877310+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:15:18.877310+0000","last_undegraded":"2024-09-05T01:15:18.877310+0000","last_fullsized":"2024-09-05T01:15:18.877310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872465+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:15:18.872465+0000","last_peered":"2024-09-05T01:15:18.872465+0000","last_clean":"2024-09-05T01:15:18.872465+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:15:18.872465+0000","last_undegraded":"2024-09-05T01:15:18.872465+0000","last_fullsized":"2024-09-05T01:15:18.872465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875893+0000","last_change":"2024-09-05T01:14:18.912776+0000","last_active":"2024-09-05T01:15:18.875893+0000","last_peered":"2024-09-05T01:15:18.875893+0000","last_clean":"2024-09-05T01:15:18.875893+0000","last_became_active":"2024-09-05T01:14:18.912337+0000","last_became_peered":"2024-09-05T01:14:18.912337+0000","last_unstale":"2024-09-05T01:15:18.875893+0000","last_undegraded":"2024-09-05T01:15:18.875893+0000","last_fullsized":"2024-09-05T01:15:18.875893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:37:57.935580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.990665+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:15:21.990665+0000","last_peered":"2024-09-05T01:15:21.990665+0000","last_clean":"2024-09-05T01:15:21.990665+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:15:21.990665+0000","last_undegraded":"2024-09-05T01:15:21.990665+0000","last_fullsized":"2024-09-05T01:15:21.990665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872383+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:15:18.872383+0000","last_peered":"2024-09-05T01:15:18.872383+0000","last_clean":"2024-09-05T01:15:18.872383+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:15:18.872383+0000","last_undegraded":"2024-09-05T01:15:18.872383+0000","last_fullsized":"2024-09-05T01:15:18.872383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872051+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:15:18.872051+0000","last_peered":"2024-09-05T01:15:18.872051+0000","last_clean":"2024-09-05T01:15:18.872051+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:15:18.872051+0000","last_undegraded":"2024-09-05T01:15:18.872051+0000","last_fullsized":"2024-09-05T01:15:18.872051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.898345+0000","last_change":"2024-09-05T01:15:18.898345+0000","last_active":"2024-09-05T01:15:18.898345+0000","last_peered":"2024-09-05T01:15:18.898345+0000","last_clean":"2024-09-05T01:15:18.898345+0000","last_became_active":"2024-09-05T01:15:18.895355+0000","last_became_peered":"2024-09-05T01:15:18.895355+0000","last_unstale":"2024-09-05T01:15:18.898345+0000","last_undegraded":"2024-09-05T01:15:18.898345+0000","last_fullsized":"2024-09-05T01:15:18.898345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:04.042332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879210+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:15:18.879210+0000","last_peered":"2024-09-05T01:15:18.879210+0000","last_clean":"2024-09-05T01:15:18.879210+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:15:18.879210+0000","last_undegraded":"2024-09-05T01:15:18.879210+0000","last_fullsized":"2024-09-05T01:15:18.879210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874602+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:15:18.874602+0000","last_peered":"2024-09-05T01:15:18.874602+0000","last_clean":"2024-09-05T01:15:18.874602+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:15:18.874602+0000","last_undegraded":"2024-09-05T01:15:18.874602+0000","last_fullsized":"2024-09-05T01:15:18.874602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894609+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:15:18.894609+0000","last_peered":"2024-09-05T01:15:18.894609+0000","last_clean":"2024-09-05T01:15:18.894609+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:15:18.894609+0000","last_undegraded":"2024-09-05T01:15:18.894609+0000","last_fullsized":"2024-09-05T01:15:18.894609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877862+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:15:18.877862+0000","last_peered":"2024-09-05T01:15:18.877862+0000","last_clean":"2024-09-05T01:15:18.877862+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:15:18.877862+0000","last_undegraded":"2024-09-05T01:15:18.877862+0000","last_fullsized":"2024-09-05T01:15:18.877862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877578+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:15:18.877578+0000","last_peered":"2024-09-05T01:15:18.877578+0000","last_clean":"2024-09-05T01:15:18.877578+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:15:18.877578+0000","last_undegraded":"2024-09-05T01:15:18.877578+0000","last_fullsized":"2024-09-05T01:15:18.877578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877709+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:15:18.877709+0000","last_peered":"2024-09-05T01:15:18.877709+0000","last_clean":"2024-09-05T01:15:18.877709+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:15:18.877709+0000","last_undegraded":"2024-09-05T01:15:18.877709+0000","last_fullsized":"2024-09-05T01:15:18.877709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876002+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:15:18.876002+0000","last_peered":"2024-09-05T01:15:18.876002+0000","last_clean":"2024-09-05T01:15:18.876002+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:15:18.876002+0000","last_undegraded":"2024-09-05T01:15:18.876002+0000","last_fullsized":"2024-09-05T01:15:18.876002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872919+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:15:18.872919+0000","last_peered":"2024-09-05T01:15:18.872919+0000","last_clean":"2024-09-05T01:15:18.872919+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:15:18.872919+0000","last_undegraded":"2024-09-05T01:15:18.872919+0000","last_fullsized":"2024-09-05T01:15:18.872919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877435+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:15:18.877435+0000","last_peered":"2024-09-05T01:15:18.877435+0000","last_clean":"2024-09-05T01:15:18.877435+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:15:18.877435+0000","last_undegraded":"2024-09-05T01:15:18.877435+0000","last_fullsized":"2024-09-05T01:15:18.877435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879901+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:15:18.879901+0000","last_peered":"2024-09-05T01:15:18.879901+0000","last_clean":"2024-09-05T01:15:18.879901+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:15:18.879901+0000","last_undegraded":"2024-09-05T01:15:18.879901+0000","last_fullsized":"2024-09-05T01:15:18.879901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892762+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:15:18.892762+0000","last_peered":"2024-09-05T01:15:18.892762+0000","last_clean":"2024-09-05T01:15:18.892762+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:15:18.892762+0000","last_undegraded":"2024-09-05T01:15:18.892762+0000","last_fullsized":"2024-09-05T01:15:18.892762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877936+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:15:18.877936+0000","last_peered":"2024-09-05T01:15:18.877936+0000","last_clean":"2024-09-05T01:15:18.877936+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:15:18.877936+0000","last_undegraded":"2024-09-05T01:15:18.877936+0000","last_fullsized":"2024-09-05T01:15:18.877936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879564+0000","last_change":"2024-09-05T01:15:18.879564+0000","last_active":"2024-09-05T01:15:18.879564+0000","last_peered":"2024-09-05T01:15:18.879564+0000","last_clean":"2024-09-05T01:15:18.879564+0000","last_became_active":"2024-09-05T01:15:18.876215+0000","last_became_peered":"2024-09-05T01:15:18.876215+0000","last_unstale":"2024-09-05T01:15:18.879564+0000","last_undegraded":"2024-09-05T01:15:18.879564+0000","last_fullsized":"2024-09-05T01:15:18.879564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:54:11.502597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877679+0000","last_change":"2024-09-05T01:14:18.913738+0000","last_active":"2024-09-05T01:15:18.877679+0000","last_peered":"2024-09-05T01:15:18.877679+0000","last_clean":"2024-09-05T01:15:18.877679+0000","last_became_active":"2024-09-05T01:14:18.913084+0000","last_became_peered":"2024-09-05T01:14:18.913084+0000","last_unstale":"2024-09-05T01:15:18.877679+0000","last_undegraded":"2024-09-05T01:15:18.877679+0000","last_fullsized":"2024-09-05T01:15:18.877679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:43.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879092+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:15:18.879092+0000","last_peered":"2024-09-05T01:15:18.879092+0000","last_clean":"2024-09-05T01:15:18.879092+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:15:18.879092+0000","last_undegraded":"2024-09-05T01:15:18.879092+0000","last_fullsized":"2024-09-05T01:15:18.879092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.880937+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:15:18.880937+0000","last_peered":"2024-09-05T01:15:18.880937+0000","last_clean":"2024-09-05T01:15:18.880937+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:15:18.880937+0000","last_undegraded":"2024-09-05T01:15:18.880937+0000","last_fullsized":"2024-09-05T01:15:18.880937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876614+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:15:18.876614+0000","last_peered":"2024-09-05T01:15:18.876614+0000","last_clean":"2024-09-05T01:15:18.876614+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:15:18.876614+0000","last_undegraded":"2024-09-05T01:15:18.876614+0000","last_fullsized":"2024-09-05T01:15:18.876614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876672+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:15:18.876672+0000","last_peered":"2024-09-05T01:15:18.876672+0000","last_clean":"2024-09-05T01:15:18.876672+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:15:18.876672+0000","last_undegraded":"2024-09-05T01:15:18.876672+0000","last_fullsized":"2024-09-05T01:15:18.876672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878035+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:15:18.878035+0000","last_peered":"2024-09-05T01:15:18.878035+0000","last_clean":"2024-09-05T01:15:18.878035+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:15:18.878035+0000","last_undegraded":"2024-09-05T01:15:18.878035+0000","last_fullsized":"2024-09-05T01:15:18.878035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872466+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:15:18.872466+0000","last_peered":"2024-09-05T01:15:18.872466+0000","last_clean":"2024-09-05T01:15:18.872466+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:15:18.872466+0000","last_undegraded":"2024-09-05T01:15:18.872466+0000","last_fullsized":"2024-09-05T01:15:18.872466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877066+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:15:18.877066+0000","last_peered":"2024-09-05T01:15:18.877066+0000","last_clean":"2024-09-05T01:15:18.877066+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:15:18.877066+0000","last_undegraded":"2024-09-05T01:15:18.877066+0000","last_fullsized":"2024-09-05T01:15:18.877066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":144,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:25.997792+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:15:25.997792+0000","last_peered":"2024-09-05T01:15:25.997792+0000","last_clean":"2024-09-05T01:15:25.997792+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:15:25.997792+0000","last_undegraded":"2024-09-05T01:15:25.997792+0000","last_fullsized":"2024-09-05T01:15:25.997792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1067,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876080+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:15:18.876080+0000","last_peered":"2024-09-05T01:15:18.876080+0000","last_clean":"2024-09-05T01:15:18.876080+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:15:18.876080+0000","last_undegraded":"2024-09-05T01:15:18.876080+0000","last_fullsized":"2024-09-05T01:15:18.876080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877281+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:15:18.877281+0000","last_peered":"2024-09-05T01:15:18.877281+0000","last_clean":"2024-09-05T01:15:18.877281+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:15:18.877281+0000","last_undegraded":"2024-09-05T01:15:18.877281+0000","last_fullsized":"2024-09-05T01:15:18.877281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877183+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:15:18.877183+0000","last_peered":"2024-09-05T01:15:18.877183+0000","last_clean":"2024-09-05T01:15:18.877183+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:15:18.877183+0000","last_undegraded":"2024-09-05T01:15:18.877183+0000","last_fullsized":"2024-09-05T01:15:18.877183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877256+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:15:18.877256+0000","last_peered":"2024-09-05T01:15:18.877256+0000","last_clean":"2024-09-05T01:15:18.877256+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:15:18.877256+0000","last_undegraded":"2024-09-05T01:15:18.877256+0000","last_fullsized":"2024-09-05T01:15:18.877256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872199+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:15:18.872199+0000","last_peered":"2024-09-05T01:15:18.872199+0000","last_clean":"2024-09-05T01:15:18.872199+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:15:18.872199+0000","last_undegraded":"2024-09-05T01:15:18.872199+0000","last_fullsized":"2024-09-05T01:15:18.872199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875174+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:15:18.875174+0000","last_peered":"2024-09-05T01:15:18.875174+0000","last_clean":"2024-09-05T01:15:18.875174+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:15:18.875174+0000","last_undegraded":"2024-09-05T01:15:18.875174+0000","last_fullsized":"2024-09-05T01:15:18.875174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879784+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:15:18.879784+0000","last_peered":"2024-09-05T01:15:18.879784+0000","last_clean":"2024-09-05T01:15:18.879784+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:15:18.879784+0000","last_undegraded":"2024-09-05T01:15:18.879784+0000","last_fullsized":"2024-09-05T01:15:18.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877835+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:15:18.877835+0000","last_peered":"2024-09-05T01:15:18.877835+0000","last_clean":"2024-09-05T01:15:18.877835+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:15:18.877835+0000","last_undegraded":"2024-09-05T01:15:18.877835+0000","last_fullsized":"2024-09-05T01:15:18.877835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":287,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.904041+0000","last_change":"2024-09-05T01:15:18.902797+0000","last_active":"2024-09-05T01:15:21.904041+0000","last_peered":"2024-09-05T01:15:21.904041+0000","last_clean":"2024-09-05T01:15:21.904041+0000","last_became_active":"2024-09-05T01:15:18.896064+0000","last_became_peered":"2024-09-05T01:15:18.896064+0000","last_unstale":"2024-09-05T01:15:21.904041+0000","last_undegraded":"2024-09-05T01:15:21.904041+0000","last_fullsized":"2024-09-05T01:15:21.904041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:53.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62325,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":159,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62325,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874983+0000","last_change":"2024-09-05T01:14:18.840367+0000","last_active":"2024-09-05T01:15:18.874983+0000","last_peered":"2024-09-05T01:15:18.874983+0000","last_clean":"2024-09-05T01:15:18.874983+0000","last_became_active":"2024-09-05T01:14:18.839937+0000","last_became_peered":"2024-09-05T01:14:18.839937+0000","last_unstale":"2024-09-05T01:15:18.874983+0000","last_undegraded":"2024-09-05T01:15:18.874983+0000","last_fullsized":"2024-09-05T01:15:18.874983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:49:14.286631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873258+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:15:18.873258+0000","last_peered":"2024-09-05T01:15:18.873258+0000","last_clean":"2024-09-05T01:15:18.873258+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:15:18.873258+0000","last_undegraded":"2024-09-05T01:15:18.873258+0000","last_fullsized":"2024-09-05T01:15:18.873258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879574+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:15:18.879574+0000","last_peered":"2024-09-05T01:15:18.879574+0000","last_clean":"2024-09-05T01:15:18.879574+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:15:18.879574+0000","last_undegraded":"2024-09-05T01:15:18.879574+0000","last_fullsized":"2024-09-05T01:15:18.879574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894131+0000","last_change":"2024-09-05T01:14:18.913011+0000","last_active":"2024-09-05T01:15:18.894131+0000","last_peered":"2024-09-05T01:15:18.894131+0000","last_clean":"2024-09-05T01:15:18.894131+0000","last_became_active":"2024-09-05T01:14:18.912546+0000","last_became_peered":"2024-09-05T01:14:18.912546+0000","last_unstale":"2024-09-05T01:15:18.894131+0000","last_undegraded":"2024-09-05T01:15:18.894131+0000","last_fullsized":"2024-09-05T01:15:18.894131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:11:02.202754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893072+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:15:18.893072+0000","last_peered":"2024-09-05T01:15:18.893072+0000","last_clean":"2024-09-05T01:15:18.893072+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:15:18.893072+0000","last_undegraded":"2024-09-05T01:15:18.893072+0000","last_fullsized":"2024-09-05T01:15:18.893072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877804+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:15:18.877804+0000","last_peered":"2024-09-05T01:15:18.877804+0000","last_clean":"2024-09-05T01:15:18.877804+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:15:18.877804+0000","last_undegraded":"2024-09-05T01:15:18.877804+0000","last_fullsized":"2024-09-05T01:15:18.877804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875496+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:15:18.875496+0000","last_peered":"2024-09-05T01:15:18.875496+0000","last_clean":"2024-09-05T01:15:18.875496+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:15:18.875496+0000","last_undegraded":"2024-09-05T01:15:18.875496+0000","last_fullsized":"2024-09-05T01:15:18.875496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873793+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:15:18.873793+0000","last_peered":"2024-09-05T01:15:18.873793+0000","last_clean":"2024-09-05T01:15:18.873793+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:15:18.873793+0000","last_undegraded":"2024-09-05T01:15:18.873793+0000","last_fullsized":"2024-09-05T01:15:18.873793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:20.892189+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:15:20.892189+0000","last_peered":"2024-09-05T01:15:20.892189+0000","last_clean":"2024-09-05T01:15:20.892189+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:15:20.892189+0000","last_undegraded":"2024-09-05T01:15:20.892189+0000","last_fullsized":"2024-09-05T01:15:20.892189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874862+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:15:18.874862+0000","last_peered":"2024-09-05T01:15:18.874862+0000","last_clean":"2024-09-05T01:15:18.874862+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:15:18.874862+0000","last_undegraded":"2024-09-05T01:15:18.874862+0000","last_fullsized":"2024-09-05T01:15:18.874862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892843+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:15:18.892843+0000","last_peered":"2024-09-05T01:15:18.892843+0000","last_clean":"2024-09-05T01:15:18.892843+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:15:18.892843+0000","last_undegraded":"2024-09-05T01:15:18.892843+0000","last_fullsized":"2024-09-05T01:15:18.892843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875596+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:15:18.875596+0000","last_peered":"2024-09-05T01:15:18.875596+0000","last_clean":"2024-09-05T01:15:18.875596+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:15:18.875596+0000","last_undegraded":"2024-09-05T01:15:18.875596+0000","last_fullsized":"2024-09-05T01:15:18.875596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876372+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:15:18.876372+0000","last_peered":"2024-09-05T01:15:18.876372+0000","last_clean":"2024-09-05T01:15:18.876372+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:15:18.876372+0000","last_undegraded":"2024-09-05T01:15:18.876372+0000","last_fullsized":"2024-09-05T01:15:18.876372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879805+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:15:18.879805+0000","last_peered":"2024-09-05T01:15:18.879805+0000","last_clean":"2024-09-05T01:15:18.879805+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:15:18.879805+0000","last_undegraded":"2024-09-05T01:15:18.879805+0000","last_fullsized":"2024-09-05T01:15:18.879805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874509+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:15:18.874509+0000","last_peered":"2024-09-05T01:15:18.874509+0000","last_clean":"2024-09-05T01:15:18.874509+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:15:18.874509+0000","last_undegraded":"2024-09-05T01:15:18.874509+0000","last_fullsized":"2024-09-05T01:15:18.874509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872057+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:15:18.872057+0000","last_peered":"2024-09-05T01:15:18.872057+0000","last_clean":"2024-09-05T01:15:18.872057+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:15:18.872057+0000","last_undegraded":"2024-09-05T01:15:18.872057+0000","last_fullsized":"2024-09-05T01:15:18.872057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875691+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:15:18.875691+0000","last_peered":"2024-09-05T01:15:18.875691+0000","last_clean":"2024-09-05T01:15:18.875691+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:15:18.875691+0000","last_undegraded":"2024-09-05T01:15:18.875691+0000","last_fullsized":"2024-09-05T01:15:18.875691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875643+0000","last_change":"2024-09-05T01:14:18.912517+0000","last_active":"2024-09-05T01:15:18.875643+0000","last_peered":"2024-09-05T01:15:18.875643+0000","last_clean":"2024-09-05T01:15:18.875643+0000","last_became_active":"2024-09-05T01:14:18.912074+0000","last_became_peered":"2024-09-05T01:14:18.912074+0000","last_unstale":"2024-09-05T01:15:18.875643+0000","last_undegraded":"2024-09-05T01:15:18.875643+0000","last_fullsized":"2024-09-05T01:15:18.875643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:16.172775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874931+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:15:18.874931+0000","last_peered":"2024-09-05T01:15:18.874931+0000","last_clean":"2024-09-05T01:15:18.874931+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:15:18.874931+0000","last_undegraded":"2024-09-05T01:15:18.874931+0000","last_fullsized":"2024-09-05T01:15:18.874931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:23.898003+0000","last_change":"2024-09-05T01:15:18.891283+0000","last_active":"2024-09-05T01:15:23.898003+0000","last_peered":"2024-09-05T01:15:23.898003+0000","last_clean":"2024-09-05T01:15:23.898003+0000","last_became_active":"2024-09-05T01:15:18.882289+0000","last_became_peered":"2024-09-05T01:15:18.882289+0000","last_unstale":"2024-09-05T01:15:23.898003+0000","last_undegraded":"2024-09-05T01:15:23.898003+0000","last_fullsized":"2024-09-05T01:15:23.898003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:45:12.506629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878425+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:15:18.878425+0000","last_peered":"2024-09-05T01:15:18.878425+0000","last_clean":"2024-09-05T01:15:18.878425+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:15:18.878425+0000","last_undegraded":"2024-09-05T01:15:18.878425+0000","last_fullsized":"2024-09-05T01:15:18.878425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875180+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:15:18.875180+0000","last_peered":"2024-09-05T01:15:18.875180+0000","last_clean":"2024-09-05T01:15:18.875180+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:15:18.875180+0000","last_undegraded":"2024-09-05T01:15:18.875180+0000","last_fullsized":"2024-09-05T01:15:18.875180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877663+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:15:18.877663+0000","last_peered":"2024-09-05T01:15:18.877663+0000","last_clean":"2024-09-05T01:15:18.877663+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:15:18.877663+0000","last_undegraded":"2024-09-05T01:15:18.877663+0000","last_fullsized":"2024-09-05T01:15:18.877663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875455+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:15:18.875455+0000","last_peered":"2024-09-05T01:15:18.875455+0000","last_clean":"2024-09-05T01:15:18.875455+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:15:18.875455+0000","last_undegraded":"2024-09-05T01:15:18.875455+0000","last_fullsized":"2024-09-05T01:15:18.875455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874866+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:15:18.874866+0000","last_peered":"2024-09-05T01:15:18.874866+0000","last_clean":"2024-09-05T01:15:18.874866+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:15:18.874866+0000","last_undegraded":"2024-09-05T01:15:18.874866+0000","last_fullsized":"2024-09-05T01:15:18.874866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892838+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:15:18.892838+0000","last_peered":"2024-09-05T01:15:18.892838+0000","last_clean":"2024-09-05T01:15:18.892838+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:15:18.892838+0000","last_undegraded":"2024-09-05T01:15:18.892838+0000","last_fullsized":"2024-09-05T01:15:18.892838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872553+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:15:18.872553+0000","last_peered":"2024-09-05T01:15:18.872553+0000","last_clean":"2024-09-05T01:15:18.872553+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:15:18.872553+0000","last_undegraded":"2024-09-05T01:15:18.872553+0000","last_fullsized":"2024-09-05T01:15:18.872553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878714+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:15:18.878714+0000","last_peered":"2024-09-05T01:15:18.878714+0000","last_clean":"2024-09-05T01:15:18.878714+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:15:18.878714+0000","last_undegraded":"2024-09-05T01:15:18.878714+0000","last_fullsized":"2024-09-05T01:15:18.878714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876503+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:15:18.876503+0000","last_peered":"2024-09-05T01:15:18.876503+0000","last_clean":"2024-09-05T01:15:18.876503+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:15:18.876503+0000","last_undegraded":"2024-09-05T01:15:18.876503+0000","last_fullsized":"2024-09-05T01:15:18.876503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875119+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:15:18.875119+0000","last_peered":"2024-09-05T01:15:18.875119+0000","last_clean":"2024-09-05T01:15:18.875119+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:15:18.875119+0000","last_undegraded":"2024-09-05T01:15:18.875119+0000","last_fullsized":"2024-09-05T01:15:18.875119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875850+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:15:18.875850+0000","last_peered":"2024-09-05T01:15:18.875850+0000","last_clean":"2024-09-05T01:15:18.875850+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:15:18.875850+0000","last_undegraded":"2024-09-05T01:15:18.875850+0000","last_fullsized":"2024-09-05T01:15:18.875850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874525+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:15:18.874525+0000","last_peered":"2024-09-05T01:15:18.874525+0000","last_clean":"2024-09-05T01:15:18.874525+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:15:18.874525+0000","last_undegraded":"2024-09-05T01:15:18.874525+0000","last_fullsized":"2024-09-05T01:15:18.874525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873202+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:15:18.873202+0000","last_peered":"2024-09-05T01:15:18.873202+0000","last_clean":"2024-09-05T01:15:18.873202+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:15:18.873202+0000","last_undegraded":"2024-09-05T01:15:18.873202+0000","last_fullsized":"2024-09-05T01:15:18.873202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872496+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:15:18.872496+0000","last_peered":"2024-09-05T01:15:18.872496+0000","last_clean":"2024-09-05T01:15:18.872496+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:15:18.872496+0000","last_undegraded":"2024-09-05T01:15:18.872496+0000","last_fullsized":"2024-09-05T01:15:18.872496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877437+0000","last_change":"2024-09-05T01:14:18.840355+0000","last_active":"2024-09-05T01:15:18.877437+0000","last_peered":"2024-09-05T01:15:18.877437+0000","last_clean":"2024-09-05T01:15:18.877437+0000","last_became_active":"2024-09-05T01:14:18.839887+0000","last_became_peered":"2024-09-05T01:14:18.839887+0000","last_unstale":"2024-09-05T01:15:18.877437+0000","last_undegraded":"2024-09-05T01:15:18.877437+0000","last_fullsized":"2024-09-05T01:15:18.877437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:49.152575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875539+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:15:18.875539+0000","last_peered":"2024-09-05T01:15:18.875539+0000","last_clean":"2024-09-05T01:15:18.875539+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:15:18.875539+0000","last_undegraded":"2024-09-05T01:15:18.875539+0000","last_fullsized":"2024-09-05T01:15:18.875539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876230+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:15:18.876230+0000","last_peered":"2024-09-05T01:15:18.876230+0000","last_clean":"2024-09-05T01:15:18.876230+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:15:18.876230+0000","last_undegraded":"2024-09-05T01:15:18.876230+0000","last_fullsized":"2024-09-05T01:15:18.876230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879606+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:15:18.879606+0000","last_peered":"2024-09-05T01:15:18.879606+0000","last_clean":"2024-09-05T01:15:18.879606+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:15:18.879606+0000","last_undegraded":"2024-09-05T01:15:18.879606+0000","last_fullsized":"2024-09-05T01:15:18.879606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872781+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:15:18.872781+0000","last_peered":"2024-09-05T01:15:18.872781+0000","last_clean":"2024-09-05T01:15:18.872781+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:15:18.872781+0000","last_undegraded":"2024-09-05T01:15:18.872781+0000","last_fullsized":"2024-09-05T01:15:18.872781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892640+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:15:18.892640+0000","last_peered":"2024-09-05T01:15:18.892640+0000","last_clean":"2024-09-05T01:15:18.892640+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:15:18.892640+0000","last_undegraded":"2024-09-05T01:15:18.892640+0000","last_fullsized":"2024-09-05T01:15:18.892640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872317+0000","last_change":"2024-09-05T01:14:18.836841+0000","last_active":"2024-09-05T01:15:18.872317+0000","last_peered":"2024-09-05T01:15:18.872317+0000","last_clean":"2024-09-05T01:15:18.872317+0000","last_became_active":"2024-09-05T01:14:18.836464+0000","last_became_peered":"2024-09-05T01:14:18.836464+0000","last_unstale":"2024-09-05T01:15:18.872317+0000","last_undegraded":"2024-09-05T01:15:18.872317+0000","last_fullsized":"2024-09-05T01:15:18.872317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:21:02.858269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894772+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:15:18.894772+0000","last_peered":"2024-09-05T01:15:18.894772+0000","last_clean":"2024-09-05T01:15:18.894772+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:15:18.894772+0000","last_undegraded":"2024-09-05T01:15:18.894772+0000","last_fullsized":"2024-09-05T01:15:18.894772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877636+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:15:18.877636+0000","last_peered":"2024-09-05T01:15:18.877636+0000","last_clean":"2024-09-05T01:15:18.877636+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:15:18.877636+0000","last_undegraded":"2024-09-05T01:15:18.877636+0000","last_fullsized":"2024-09-05T01:15:18.877636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875974+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:15:18.875974+0000","last_peered":"2024-09-05T01:15:18.875974+0000","last_clean":"2024-09-05T01:15:18.875974+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:15:18.875974+0000","last_undegraded":"2024-09-05T01:15:18.875974+0000","last_fullsized":"2024-09-05T01:15:18.875974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873830+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:15:18.873830+0000","last_peered":"2024-09-05T01:15:18.873830+0000","last_clean":"2024-09-05T01:15:18.873830+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:15:18.873830+0000","last_undegraded":"2024-09-05T01:15:18.873830+0000","last_fullsized":"2024-09-05T01:15:18.873830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874356+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:15:18.874356+0000","last_peered":"2024-09-05T01:15:18.874356+0000","last_clean":"2024-09-05T01:15:18.874356+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:15:18.874356+0000","last_undegraded":"2024-09-05T01:15:18.874356+0000","last_fullsized":"2024-09-05T01:15:18.874356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877120+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:15:18.877120+0000","last_peered":"2024-09-05T01:15:18.877120+0000","last_clean":"2024-09-05T01:15:18.877120+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:15:18.877120+0000","last_undegraded":"2024-09-05T01:15:18.877120+0000","last_fullsized":"2024-09-05T01:15:18.877120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875187+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:15:18.875187+0000","last_peered":"2024-09-05T01:15:18.875187+0000","last_clean":"2024-09-05T01:15:18.875187+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:15:18.875187+0000","last_undegraded":"2024-09-05T01:15:18.875187+0000","last_fullsized":"2024-09-05T01:15:18.875187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879887+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:15:18.879887+0000","last_peered":"2024-09-05T01:15:18.879887+0000","last_clean":"2024-09-05T01:15:18.879887+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:15:18.879887+0000","last_undegraded":"2024-09-05T01:15:18.879887+0000","last_fullsized":"2024-09-05T01:15:18.879887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894875+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:15:18.894875+0000","last_peered":"2024-09-05T01:15:18.894875+0000","last_clean":"2024-09-05T01:15:18.894875+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:15:18.894875+0000","last_undegraded":"2024-09-05T01:15:18.894875+0000","last_fullsized":"2024-09-05T01:15:18.894875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892807+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:15:18.892807+0000","last_peered":"2024-09-05T01:15:18.892807+0000","last_clean":"2024-09-05T01:15:18.892807+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:15:18.892807+0000","last_undegraded":"2024-09-05T01:15:18.892807+0000","last_fullsized":"2024-09-05T01:15:18.892807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879435+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:15:18.879435+0000","last_peered":"2024-09-05T01:15:18.879435+0000","last_clean":"2024-09-05T01:15:18.879435+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:15:18.879435+0000","last_undegraded":"2024-09-05T01:15:18.879435+0000","last_fullsized":"2024-09-05T01:15:18.879435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67427,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":594,"num_read_kb":369,"num_write":176,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":209508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":516334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1724416,"data_stored":1164292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":519031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201337,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":514855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1642496,"data_stored":1111865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593795,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":510111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1105339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":585271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":510305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1048576,"data_stored":511905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":572825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":511499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:15:28.619 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T01:15:28.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T01:15:28.867 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:29.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2280606485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd524106f50 msgr2=0x7fd52410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:29.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2280606485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd524106f50 0x7fd52410eaf0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009cd0 tx=0x7fd51c02a8e0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2280606485 shutdown_connections 2024-09-05T01:15:29.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2280606485 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd524106f50 0x7fd52410eaf0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2280606485 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524100ca0 0x7fd524106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2280606485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd5241002f0 0x7fd5241006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2280606485 >> 172.21.15.129:0/2280606485 conn(0x7fd524077210 msgr2=0x7fd5240fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2280606485 shutdown_connections 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.357+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2280606485 wait complete. 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 Processor -- start 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 -- start start 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 0x7fd5241017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524106f50 0x7fd524101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd524111eb0 con 0x7fd5241002f0 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd524111d30 con 0x7fd524100ca0 2024-09-05T01:15:29.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd52a1bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd524112030 con 0x7fd524106f50 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52240/0 (socket says 172.21.15.129:52240) 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 -- 172.21.15.129:0/2931047357 learned_addr learned my addr 172.21.15.129:0/2931047357 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd522ffd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 0x7fd5241017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 -- 172.21.15.129:0/2931047357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524106f50 msgr2=0x7fd524101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd523fff640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524106f50 0x7fd524101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524106f50 0x7fd524101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 -- 172.21.15.129:0/2931047357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 msgr2=0x7fd5241017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 0x7fd5241017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.358+0000 7fd5237fe640 1 -- 172.21.15.129:0/2931047357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5241a68d0 con 0x7fd5241002f0 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd522ffd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 0x7fd5241017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd5237fe640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd51400d950 tx=0x7fd51400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:29.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514017030 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd514004020 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5241a6bc0 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5140047a0 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.359+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5241a7100 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.360+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5241006d0 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.361+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd5140041d0 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.362+0000 7fd520ff9640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 0x7fd5100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.362+0000 7fd522ffd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 0x7fd5100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.362+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fd5140cf110 con 0x7fd5241002f0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.362+0000 7fd522ffd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 0x7fd5100a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd524102d00 tx=0x7fd50c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:29.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.366+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd51409bb00 con 0x7fd5241002f0 2024-09-05T01:15:29.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.530+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 --> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd524062630 con 0x7fd5100a2c10 2024-09-05T01:15:29.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.538+0000 7fd520ff9640 1 -- 172.21.15.129:0/2931047357 <== mgr.14150 v2:172.21.15.129:6800/655841650 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340516 (secure 0 0 0) 0x7fd524062630 con 0x7fd5100a2c10 2024-09-05T01:15:29.541 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:29.543 INFO:teuthology.orchestra.run.smithi129.stderr:dumped all 2024-09-05T01:15:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 msgr2=0x7fd5100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 0x7fd5100a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd524102d00 tx=0x7fd50c008040 comp rx=0 tx=0).stop 2024-09-05T01:15:29.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 msgr2=0x7fd524101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd51400d950 tx=0x7fd51400de10 comp rx=0 tx=0).stop 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 shutdown_connections 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5100a2c10 0x7fd5100a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd524106f50 0x7fd524101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd524100ca0 0x7fd5241017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 --2- 172.21.15.129:0/2931047357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd5241002f0 0x7fd524101270 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 >> 172.21.15.129:0/2931047357 conn(0x7fd524077210 msgr2=0x7fd5241051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 shutdown_connections 2024-09-05T01:15:29.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:29.541+0000 7fd52a1bd640 1 -- 172.21.15.129:0/2931047357 wait complete. 2024-09-05T01:15:30.201 INFO:teuthology.orchestra.run.smithi129.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-05T01:15:27.783121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658029,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":720,"num_read_kb":478,"num_write":359,"num_write_kb":3063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1244217,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":789,"ondisk_log_size":789,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":401008,"kb_used_data":14196,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124540816,"statfs":{"total":1151940427776,"available":1151529795584,"internally_reserved":0,"allocated":14536704,"data_stored":8133632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":395752763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":958,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":22,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.913416"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876259+0000","last_change":"2024-09-05T01:13:03.879606+0000","last_active":"2024-09-05T01:15:18.876259+0000","last_peered":"2024-09-05T01:15:18.876259+0000","last_clean":"2024-09-05T01:15:18.876259+0000","last_became_active":"2024-09-05T01:13:03.879303+0000","last_became_peered":"2024-09-05T01:13:03.879303+0000","last_unstale":"2024-09-05T01:15:18.876259+0000","last_undegraded":"2024-09-05T01:15:18.876259+0000","last_fullsized":"2024-09-05T01:15:18.876259+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:33:45.350527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874746+0000","last_change":"2024-09-05T01:13:05.894169+0000","last_active":"2024-09-05T01:15:18.874746+0000","last_peered":"2024-09-05T01:15:18.874746+0000","last_clean":"2024-09-05T01:15:18.874746+0000","last_became_active":"2024-09-05T01:13:05.893820+0000","last_became_peered":"2024-09-05T01:13:05.893820+0000","last_unstale":"2024-09-05T01:15:18.874746+0000","last_undegraded":"2024-09-05T01:15:18.874746+0000","last_fullsized":"2024-09-05T01:15:18.874746+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:14:09.356186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874793+0000","last_change":"2024-09-05T01:13:12.923079+0000","last_active":"2024-09-05T01:15:18.874793+0000","last_peered":"2024-09-05T01:15:18.874793+0000","last_clean":"2024-09-05T01:15:18.874793+0000","last_became_active":"2024-09-05T01:13:12.922645+0000","last_became_peered":"2024-09-05T01:13:12.922645+0000","last_unstale":"2024-09-05T01:15:18.874793+0000","last_undegraded":"2024-09-05T01:15:18.874793+0000","last_fullsized":"2024-09-05T01:15:18.874793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:55:17.659040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875030+0000","last_change":"2024-09-05T01:13:03.966872+0000","last_active":"2024-09-05T01:15:18.875030+0000","last_peered":"2024-09-05T01:15:18.875030+0000","last_clean":"2024-09-05T01:15:18.875030+0000","last_became_active":"2024-09-05T01:13:03.966484+0000","last_became_peered":"2024-09-05T01:13:03.966484+0000","last_unstale":"2024-09-05T01:15:18.875030+0000","last_undegraded":"2024-09-05T01:15:18.875030+0000","last_fullsized":"2024-09-05T01:15:18.875030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:23:06.915847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893074+0000","last_change":"2024-09-05T01:13:05.892061+0000","last_active":"2024-09-05T01:15:18.893074+0000","last_peered":"2024-09-05T01:15:18.893074+0000","last_clean":"2024-09-05T01:15:18.893074+0000","last_became_active":"2024-09-05T01:13:05.891786+0000","last_became_peered":"2024-09-05T01:13:05.891786+0000","last_unstale":"2024-09-05T01:15:18.893074+0000","last_undegraded":"2024-09-05T01:15:18.893074+0000","last_fullsized":"2024-09-05T01:15:18.893074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:02:46.478602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894431+0000","last_change":"2024-09-05T01:14:18.913937+0000","last_active":"2024-09-05T01:15:18.894431+0000","last_peered":"2024-09-05T01:15:18.894431+0000","last_clean":"2024-09-05T01:15:18.894431+0000","last_became_active":"2024-09-05T01:14:18.913425+0000","last_became_peered":"2024-09-05T01:14:18.913425+0000","last_unstale":"2024-09-05T01:15:18.894431+0000","last_undegraded":"2024-09-05T01:15:18.894431+0000","last_fullsized":"2024-09-05T01:15:18.894431+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:56.174787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878167+0000","last_change":"2024-09-05T01:13:03.879590+0000","last_active":"2024-09-05T01:15:18.878167+0000","last_peered":"2024-09-05T01:15:18.878167+0000","last_clean":"2024-09-05T01:15:18.878167+0000","last_became_active":"2024-09-05T01:13:03.879118+0000","last_became_peered":"2024-09-05T01:13:03.879118+0000","last_unstale":"2024-09-05T01:15:18.878167+0000","last_undegraded":"2024-09-05T01:15:18.878167+0000","last_fullsized":"2024-09-05T01:15:18.878167+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:26.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873242+0000","last_change":"2024-09-05T01:13:05.884542+0000","last_active":"2024-09-05T01:15:18.873242+0000","last_peered":"2024-09-05T01:15:18.873242+0000","last_clean":"2024-09-05T01:15:18.873242+0000","last_became_active":"2024-09-05T01:13:05.884204+0000","last_became_peered":"2024-09-05T01:13:05.884204+0000","last_unstale":"2024-09-05T01:15:18.873242+0000","last_undegraded":"2024-09-05T01:15:18.873242+0000","last_fullsized":"2024-09-05T01:15:18.873242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:24:17.848030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894484+0000","last_change":"2024-09-05T01:13:12.921963+0000","last_active":"2024-09-05T01:15:18.894484+0000","last_peered":"2024-09-05T01:15:18.894484+0000","last_clean":"2024-09-05T01:15:18.894484+0000","last_became_active":"2024-09-05T01:13:12.921509+0000","last_became_peered":"2024-09-05T01:13:12.921509+0000","last_unstale":"2024-09-05T01:15:18.894484+0000","last_undegraded":"2024-09-05T01:15:18.894484+0000","last_fullsized":"2024-09-05T01:15:18.894484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:59:46.881110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878648+0000","last_change":"2024-09-05T01:13:03.879766+0000","last_active":"2024-09-05T01:15:18.878648+0000","last_peered":"2024-09-05T01:15:18.878648+0000","last_clean":"2024-09-05T01:15:18.878648+0000","last_became_active":"2024-09-05T01:13:03.879406+0000","last_became_peered":"2024-09-05T01:13:03.879406+0000","last_unstale":"2024-09-05T01:15:18.878648+0000","last_undegraded":"2024-09-05T01:15:18.878648+0000","last_fullsized":"2024-09-05T01:15:18.878648+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:54.754717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879364+0000","last_change":"2024-09-05T01:14:18.913351+0000","last_active":"2024-09-05T01:15:18.879364+0000","last_peered":"2024-09-05T01:15:18.879364+0000","last_clean":"2024-09-05T01:15:18.879364+0000","last_became_active":"2024-09-05T01:14:18.912935+0000","last_became_peered":"2024-09-05T01:14:18.912935+0000","last_unstale":"2024-09-05T01:15:18.879364+0000","last_undegraded":"2024-09-05T01:15:18.879364+0000","last_fullsized":"2024-09-05T01:15:18.879364+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:32:30.771816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893914+0000","last_change":"2024-09-05T01:13:12.925414+0000","last_active":"2024-09-05T01:15:18.893914+0000","last_peered":"2024-09-05T01:15:18.893914+0000","last_clean":"2024-09-05T01:15:18.893914+0000","last_became_active":"2024-09-05T01:13:12.924953+0000","last_became_peered":"2024-09-05T01:13:12.924953+0000","last_unstale":"2024-09-05T01:15:18.893914+0000","last_undegraded":"2024-09-05T01:15:18.893914+0000","last_fullsized":"2024-09-05T01:15:18.893914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:37:46.623039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877171+0000","last_change":"2024-09-05T01:13:05.897052+0000","last_active":"2024-09-05T01:15:18.877171+0000","last_peered":"2024-09-05T01:15:18.877171+0000","last_clean":"2024-09-05T01:15:18.877171+0000","last_became_active":"2024-09-05T01:13:05.896756+0000","last_became_peered":"2024-09-05T01:13:05.896756+0000","last_unstale":"2024-09-05T01:15:18.877171+0000","last_undegraded":"2024-09-05T01:15:18.877171+0000","last_fullsized":"2024-09-05T01:15:18.877171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:06:53.467459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886549+0000","last_change":"2024-09-05T01:15:18.886013+0000","last_active":"2024-09-05T01:15:18.886549+0000","last_peered":"2024-09-05T01:15:18.886549+0000","last_clean":"2024-09-05T01:15:18.886549+0000","last_became_active":"2024-09-05T01:15:18.881459+0000","last_became_peered":"2024-09-05T01:15:18.881459+0000","last_unstale":"2024-09-05T01:15:18.886549+0000","last_undegraded":"2024-09-05T01:15:18.886549+0000","last_fullsized":"2024-09-05T01:15:18.886549+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:59:40.021197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,7],"acting":[8,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894054+0000","last_change":"2024-09-05T01:13:12.924231+0000","last_active":"2024-09-05T01:15:18.894054+0000","last_peered":"2024-09-05T01:15:18.894054+0000","last_clean":"2024-09-05T01:15:18.894054+0000","last_became_active":"2024-09-05T01:13:12.923784+0000","last_became_peered":"2024-09-05T01:13:12.923784+0000","last_unstale":"2024-09-05T01:15:18.894054+0000","last_undegraded":"2024-09-05T01:15:18.894054+0000","last_fullsized":"2024-09-05T01:15:18.894054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:34:00.907430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874595+0000","last_change":"2024-09-05T01:13:05.883604+0000","last_active":"2024-09-05T01:15:18.874595+0000","last_peered":"2024-09-05T01:15:18.874595+0000","last_clean":"2024-09-05T01:15:18.874595+0000","last_became_active":"2024-09-05T01:13:05.883316+0000","last_became_peered":"2024-09-05T01:13:05.883316+0000","last_unstale":"2024-09-05T01:15:18.874595+0000","last_undegraded":"2024-09-05T01:15:18.874595+0000","last_fullsized":"2024-09-05T01:15:18.874595+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:19.029908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874871+0000","last_change":"2024-09-05T01:13:03.881635+0000","last_active":"2024-09-05T01:15:18.874871+0000","last_peered":"2024-09-05T01:15:18.874871+0000","last_clean":"2024-09-05T01:15:18.874871+0000","last_became_active":"2024-09-05T01:13:03.881269+0000","last_became_peered":"2024-09-05T01:13:03.881269+0000","last_unstale":"2024-09-05T01:15:18.874871+0000","last_undegraded":"2024-09-05T01:15:18.874871+0000","last_fullsized":"2024-09-05T01:15:18.874871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:11:23.980881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894639+0000","last_change":"2024-09-05T01:13:12.919829+0000","last_active":"2024-09-05T01:15:18.894639+0000","last_peered":"2024-09-05T01:15:18.894639+0000","last_clean":"2024-09-05T01:15:18.894639+0000","last_became_active":"2024-09-05T01:13:12.919371+0000","last_became_peered":"2024-09-05T01:13:12.919371+0000","last_unstale":"2024-09-05T01:15:18.894639+0000","last_undegraded":"2024-09-05T01:15:18.894639+0000","last_fullsized":"2024-09-05T01:15:18.894639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:47.945358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877791+0000","last_change":"2024-09-05T01:13:05.902873+0000","last_active":"2024-09-05T01:15:18.877791+0000","last_peered":"2024-09-05T01:15:18.877791+0000","last_clean":"2024-09-05T01:15:18.877791+0000","last_became_active":"2024-09-05T01:13:05.902513+0000","last_became_peered":"2024-09-05T01:13:05.902513+0000","last_unstale":"2024-09-05T01:15:18.877791+0000","last_undegraded":"2024-09-05T01:15:18.877791+0000","last_fullsized":"2024-09-05T01:15:18.877791+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:57:40.054078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874641+0000","last_change":"2024-09-05T01:13:03.881047+0000","last_active":"2024-09-05T01:15:18.874641+0000","last_peered":"2024-09-05T01:15:18.874641+0000","last_clean":"2024-09-05T01:15:18.874641+0000","last_became_active":"2024-09-05T01:13:03.880674+0000","last_became_peered":"2024-09-05T01:13:03.880674+0000","last_unstale":"2024-09-05T01:15:18.874641+0000","last_undegraded":"2024-09-05T01:15:18.874641+0000","last_fullsized":"2024-09-05T01:15:18.874641+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:30:51.011176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876013+0000","last_change":"2024-09-05T01:13:12.922930+0000","last_active":"2024-09-05T01:15:18.876013+0000","last_peered":"2024-09-05T01:15:18.876013+0000","last_clean":"2024-09-05T01:15:18.876013+0000","last_became_active":"2024-09-05T01:13:12.922378+0000","last_became_peered":"2024-09-05T01:13:12.922378+0000","last_unstale":"2024-09-05T01:15:18.876013+0000","last_undegraded":"2024-09-05T01:15:18.876013+0000","last_fullsized":"2024-09-05T01:15:18.876013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:20:47.179808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878123+0000","last_change":"2024-09-05T01:13:05.891208+0000","last_active":"2024-09-05T01:15:18.878123+0000","last_peered":"2024-09-05T01:15:18.878123+0000","last_clean":"2024-09-05T01:15:18.878123+0000","last_became_active":"2024-09-05T01:13:05.890692+0000","last_became_peered":"2024-09-05T01:13:05.890692+0000","last_unstale":"2024-09-05T01:15:18.878123+0000","last_undegraded":"2024-09-05T01:15:18.878123+0000","last_fullsized":"2024-09-05T01:15:18.878123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:48:46.341928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873055+0000","last_change":"2024-09-05T01:13:03.966921+0000","last_active":"2024-09-05T01:15:18.873055+0000","last_peered":"2024-09-05T01:15:18.873055+0000","last_clean":"2024-09-05T01:15:18.873055+0000","last_became_active":"2024-09-05T01:13:03.966453+0000","last_became_peered":"2024-09-05T01:13:03.966453+0000","last_unstale":"2024-09-05T01:15:18.873055+0000","last_undegraded":"2024-09-05T01:15:18.873055+0000","last_fullsized":"2024-09-05T01:15:18.873055+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:42:01.919635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894188+0000","last_change":"2024-09-05T01:13:12.915914+0000","last_active":"2024-09-05T01:15:18.894188+0000","last_peered":"2024-09-05T01:15:18.894188+0000","last_clean":"2024-09-05T01:15:18.894188+0000","last_became_active":"2024-09-05T01:13:12.915359+0000","last_became_peered":"2024-09-05T01:13:12.915359+0000","last_unstale":"2024-09-05T01:15:18.894188+0000","last_undegraded":"2024-09-05T01:15:18.894188+0000","last_fullsized":"2024-09-05T01:15:18.894188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:21:46.607913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892951+0000","last_change":"2024-09-05T01:14:18.912689+0000","last_active":"2024-09-05T01:15:18.892951+0000","last_peered":"2024-09-05T01:15:18.892951+0000","last_clean":"2024-09-05T01:15:18.892951+0000","last_became_active":"2024-09-05T01:14:18.912358+0000","last_became_peered":"2024-09-05T01:14:18.912358+0000","last_unstale":"2024-09-05T01:15:18.892951+0000","last_undegraded":"2024-09-05T01:15:18.892951+0000","last_fullsized":"2024-09-05T01:15:18.892951+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:18:59.876348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894446+0000","last_change":"2024-09-05T01:13:04.190302+0000","last_active":"2024-09-05T01:15:18.894446+0000","last_peered":"2024-09-05T01:15:18.894446+0000","last_clean":"2024-09-05T01:15:18.894446+0000","last_became_active":"2024-09-05T01:13:04.189829+0000","last_became_peered":"2024-09-05T01:13:04.189829+0000","last_unstale":"2024-09-05T01:15:18.894446+0000","last_undegraded":"2024-09-05T01:15:18.894446+0000","last_fullsized":"2024-09-05T01:15:18.894446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:21:58.334687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875054+0000","last_change":"2024-09-05T01:13:12.921788+0000","last_active":"2024-09-05T01:15:18.875054+0000","last_peered":"2024-09-05T01:15:18.875054+0000","last_clean":"2024-09-05T01:15:18.875054+0000","last_became_active":"2024-09-05T01:13:12.920865+0000","last_became_peered":"2024-09-05T01:13:12.920865+0000","last_unstale":"2024-09-05T01:15:18.875054+0000","last_undegraded":"2024-09-05T01:15:18.875054+0000","last_fullsized":"2024-09-05T01:15:18.875054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:31:24.995448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875149+0000","last_change":"2024-09-05T01:13:05.891579+0000","last_active":"2024-09-05T01:15:18.875149+0000","last_peered":"2024-09-05T01:15:18.875149+0000","last_clean":"2024-09-05T01:15:18.875149+0000","last_became_active":"2024-09-05T01:13:05.891199+0000","last_became_peered":"2024-09-05T01:13:05.891199+0000","last_unstale":"2024-09-05T01:15:18.875149+0000","last_undegraded":"2024-09-05T01:15:18.875149+0000","last_fullsized":"2024-09-05T01:15:18.875149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:27:17.286101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.886618+0000","last_change":"2024-09-05T01:15:18.886355+0000","last_active":"2024-09-05T01:15:18.886618+0000","last_peered":"2024-09-05T01:15:18.886618+0000","last_clean":"2024-09-05T01:15:18.886618+0000","last_became_active":"2024-09-05T01:15:18.881103+0000","last_became_peered":"2024-09-05T01:15:18.881103+0000","last_unstale":"2024-09-05T01:15:18.886618+0000","last_undegraded":"2024-09-05T01:15:18.886618+0000","last_fullsized":"2024-09-05T01:15:18.886618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:34:33.131422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874250+0000","last_change":"2024-09-05T01:13:12.917122+0000","last_active":"2024-09-05T01:15:18.874250+0000","last_peered":"2024-09-05T01:15:18.874250+0000","last_clean":"2024-09-05T01:15:18.874250+0000","last_became_active":"2024-09-05T01:13:12.916748+0000","last_became_peered":"2024-09-05T01:13:12.916748+0000","last_unstale":"2024-09-05T01:15:18.874250+0000","last_undegraded":"2024-09-05T01:15:18.874250+0000","last_fullsized":"2024-09-05T01:15:18.874250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:31:02.440957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872626+0000","last_change":"2024-09-05T01:13:05.887302+0000","last_active":"2024-09-05T01:15:18.872626+0000","last_peered":"2024-09-05T01:15:18.872626+0000","last_clean":"2024-09-05T01:15:18.872626+0000","last_became_active":"2024-09-05T01:13:05.887018+0000","last_became_peered":"2024-09-05T01:13:05.887018+0000","last_unstale":"2024-09-05T01:15:18.872626+0000","last_undegraded":"2024-09-05T01:15:18.872626+0000","last_fullsized":"2024-09-05T01:15:18.872626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:08:37.878183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878065+0000","last_change":"2024-09-05T01:13:03.965521+0000","last_active":"2024-09-05T01:15:18.878065+0000","last_peered":"2024-09-05T01:15:18.878065+0000","last_clean":"2024-09-05T01:15:18.878065+0000","last_became_active":"2024-09-05T01:13:03.965171+0000","last_became_peered":"2024-09-05T01:13:03.965171+0000","last_unstale":"2024-09-05T01:15:18.878065+0000","last_undegraded":"2024-09-05T01:15:18.878065+0000","last_fullsized":"2024-09-05T01:15:18.878065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:21:46.857530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878364+0000","last_change":"2024-09-05T01:13:12.922133+0000","last_active":"2024-09-05T01:15:18.878364+0000","last_peered":"2024-09-05T01:15:18.878364+0000","last_clean":"2024-09-05T01:15:18.878364+0000","last_became_active":"2024-09-05T01:13:12.921669+0000","last_became_peered":"2024-09-05T01:13:12.921669+0000","last_unstale":"2024-09-05T01:15:18.878364+0000","last_undegraded":"2024-09-05T01:15:18.878364+0000","last_fullsized":"2024-09-05T01:15:18.878364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:12:00.123143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875341+0000","last_change":"2024-09-05T01:13:05.884528+0000","last_active":"2024-09-05T01:15:18.875341+0000","last_peered":"2024-09-05T01:15:18.875341+0000","last_clean":"2024-09-05T01:15:18.875341+0000","last_became_active":"2024-09-05T01:13:05.884206+0000","last_became_peered":"2024-09-05T01:13:05.884206+0000","last_unstale":"2024-09-05T01:15:18.875341+0000","last_undegraded":"2024-09-05T01:15:18.875341+0000","last_fullsized":"2024-09-05T01:15:18.875341+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:10.926760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875954+0000","last_change":"2024-09-05T01:13:03.964891+0000","last_active":"2024-09-05T01:15:18.875954+0000","last_peered":"2024-09-05T01:15:18.875954+0000","last_clean":"2024-09-05T01:15:18.875954+0000","last_became_active":"2024-09-05T01:13:03.964543+0000","last_became_peered":"2024-09-05T01:13:03.964543+0000","last_unstale":"2024-09-05T01:15:18.875954+0000","last_undegraded":"2024-09-05T01:15:18.875954+0000","last_fullsized":"2024-09-05T01:15:18.875954+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:00:37.820761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878308+0000","last_change":"2024-09-05T01:13:12.922413+0000","last_active":"2024-09-05T01:15:18.878308+0000","last_peered":"2024-09-05T01:15:18.878308+0000","last_clean":"2024-09-05T01:15:18.878308+0000","last_became_active":"2024-09-05T01:13:12.922032+0000","last_became_peered":"2024-09-05T01:13:12.922032+0000","last_unstale":"2024-09-05T01:15:18.878308+0000","last_undegraded":"2024-09-05T01:15:18.878308+0000","last_fullsized":"2024-09-05T01:15:18.878308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:31:16.084883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875250+0000","last_change":"2024-09-05T01:13:05.889509+0000","last_active":"2024-09-05T01:15:18.875250+0000","last_peered":"2024-09-05T01:15:18.875250+0000","last_clean":"2024-09-05T01:15:18.875250+0000","last_became_active":"2024-09-05T01:13:05.888785+0000","last_became_peered":"2024-09-05T01:13:05.888785+0000","last_unstale":"2024-09-05T01:15:18.875250+0000","last_undegraded":"2024-09-05T01:15:18.875250+0000","last_fullsized":"2024-09-05T01:15:18.875250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:50:48.935263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877310+0000","last_change":"2024-09-05T01:13:03.879856+0000","last_active":"2024-09-05T01:15:18.877310+0000","last_peered":"2024-09-05T01:15:18.877310+0000","last_clean":"2024-09-05T01:15:18.877310+0000","last_became_active":"2024-09-05T01:13:03.879469+0000","last_became_peered":"2024-09-05T01:13:03.879469+0000","last_unstale":"2024-09-05T01:15:18.877310+0000","last_undegraded":"2024-09-05T01:15:18.877310+0000","last_fullsized":"2024-09-05T01:15:18.877310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:28:48.110536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872465+0000","last_change":"2024-09-05T01:13:12.917232+0000","last_active":"2024-09-05T01:15:18.872465+0000","last_peered":"2024-09-05T01:15:18.872465+0000","last_clean":"2024-09-05T01:15:18.872465+0000","last_became_active":"2024-09-05T01:13:12.916886+0000","last_became_peered":"2024-09-05T01:13:12.916886+0000","last_unstale":"2024-09-05T01:15:18.872465+0000","last_undegraded":"2024-09-05T01:15:18.872465+0000","last_fullsized":"2024-09-05T01:15:18.872465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:01:46.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875893+0000","last_change":"2024-09-05T01:14:18.912776+0000","last_active":"2024-09-05T01:15:18.875893+0000","last_peered":"2024-09-05T01:15:18.875893+0000","last_clean":"2024-09-05T01:15:18.875893+0000","last_became_active":"2024-09-05T01:14:18.912337+0000","last_became_peered":"2024-09-05T01:14:18.912337+0000","last_unstale":"2024-09-05T01:15:18.875893+0000","last_undegraded":"2024-09-05T01:15:18.875893+0000","last_fullsized":"2024-09-05T01:15:18.875893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:37:57.935580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.990665+0000","last_change":"2024-09-05T01:13:03.964936+0000","last_active":"2024-09-05T01:15:21.990665+0000","last_peered":"2024-09-05T01:15:21.990665+0000","last_clean":"2024-09-05T01:15:21.990665+0000","last_became_active":"2024-09-05T01:13:03.964602+0000","last_became_peered":"2024-09-05T01:13:03.964602+0000","last_unstale":"2024-09-05T01:15:21.990665+0000","last_undegraded":"2024-09-05T01:15:21.990665+0000","last_fullsized":"2024-09-05T01:15:21.990665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:56:08.920600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":95,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872383+0000","last_change":"2024-09-05T01:13:12.912762+0000","last_active":"2024-09-05T01:15:18.872383+0000","last_peered":"2024-09-05T01:15:18.872383+0000","last_clean":"2024-09-05T01:15:18.872383+0000","last_became_active":"2024-09-05T01:13:12.912384+0000","last_became_peered":"2024-09-05T01:13:12.912384+0000","last_unstale":"2024-09-05T01:15:18.872383+0000","last_undegraded":"2024-09-05T01:15:18.872383+0000","last_fullsized":"2024-09-05T01:15:18.872383+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:00.573673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872051+0000","last_change":"2024-09-05T01:13:05.889694+0000","last_active":"2024-09-05T01:15:18.872051+0000","last_peered":"2024-09-05T01:15:18.872051+0000","last_clean":"2024-09-05T01:15:18.872051+0000","last_became_active":"2024-09-05T01:13:05.889372+0000","last_became_peered":"2024-09-05T01:13:05.889372+0000","last_unstale":"2024-09-05T01:15:18.872051+0000","last_undegraded":"2024-09-05T01:15:18.872051+0000","last_fullsized":"2024-09-05T01:15:18.872051+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:55.236667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.898345+0000","last_change":"2024-09-05T01:15:18.898345+0000","last_active":"2024-09-05T01:15:18.898345+0000","last_peered":"2024-09-05T01:15:18.898345+0000","last_clean":"2024-09-05T01:15:18.898345+0000","last_became_active":"2024-09-05T01:15:18.895355+0000","last_became_peered":"2024-09-05T01:15:18.895355+0000","last_unstale":"2024-09-05T01:15:18.898345+0000","last_undegraded":"2024-09-05T01:15:18.898345+0000","last_fullsized":"2024-09-05T01:15:18.898345+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:35:04.042332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879210+0000","last_change":"2024-09-05T01:13:12.911651+0000","last_active":"2024-09-05T01:15:18.879210+0000","last_peered":"2024-09-05T01:15:18.879210+0000","last_clean":"2024-09-05T01:15:18.879210+0000","last_became_active":"2024-09-05T01:13:12.911205+0000","last_became_peered":"2024-09-05T01:13:12.911205+0000","last_unstale":"2024-09-05T01:15:18.879210+0000","last_undegraded":"2024-09-05T01:15:18.879210+0000","last_fullsized":"2024-09-05T01:15:18.879210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:37:34.279556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874602+0000","last_change":"2024-09-05T01:13:05.896133+0000","last_active":"2024-09-05T01:15:18.874602+0000","last_peered":"2024-09-05T01:15:18.874602+0000","last_clean":"2024-09-05T01:15:18.874602+0000","last_became_active":"2024-09-05T01:13:05.895836+0000","last_became_peered":"2024-09-05T01:13:05.895836+0000","last_unstale":"2024-09-05T01:15:18.874602+0000","last_undegraded":"2024-09-05T01:15:18.874602+0000","last_fullsized":"2024-09-05T01:15:18.874602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:08:07.636476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894609+0000","last_change":"2024-09-05T01:13:03.882377+0000","last_active":"2024-09-05T01:15:18.894609+0000","last_peered":"2024-09-05T01:15:18.894609+0000","last_clean":"2024-09-05T01:15:18.894609+0000","last_became_active":"2024-09-05T01:13:03.882026+0000","last_became_peered":"2024-09-05T01:13:03.882026+0000","last_unstale":"2024-09-05T01:15:18.894609+0000","last_undegraded":"2024-09-05T01:15:18.894609+0000","last_fullsized":"2024-09-05T01:15:18.894609+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:37:12.138742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877862+0000","last_change":"2024-09-05T01:11:50.930912+0000","last_active":"2024-09-05T01:15:18.877862+0000","last_peered":"2024-09-05T01:15:18.877862+0000","last_clean":"2024-09-05T01:15:18.877862+0000","last_became_active":"2024-09-05T01:11:50.930445+0000","last_became_peered":"2024-09-05T01:11:50.930445+0000","last_unstale":"2024-09-05T01:15:18.877862+0000","last_undegraded":"2024-09-05T01:15:18.877862+0000","last_fullsized":"2024-09-05T01:15:18.877862+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:08:28.441867+0000","last_clean_scrub_stamp":"2024-09-05T01:08:28.441867+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:27:38.967298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877578+0000","last_change":"2024-09-05T01:13:12.921968+0000","last_active":"2024-09-05T01:15:18.877578+0000","last_peered":"2024-09-05T01:15:18.877578+0000","last_clean":"2024-09-05T01:15:18.877578+0000","last_became_active":"2024-09-05T01:13:12.921422+0000","last_became_peered":"2024-09-05T01:13:12.921422+0000","last_unstale":"2024-09-05T01:15:18.877578+0000","last_undegraded":"2024-09-05T01:15:18.877578+0000","last_fullsized":"2024-09-05T01:15:18.877578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:04:37.653772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877709+0000","last_change":"2024-09-05T01:13:05.897174+0000","last_active":"2024-09-05T01:15:18.877709+0000","last_peered":"2024-09-05T01:15:18.877709+0000","last_clean":"2024-09-05T01:15:18.877709+0000","last_became_active":"2024-09-05T01:13:05.896886+0000","last_became_peered":"2024-09-05T01:13:05.896886+0000","last_unstale":"2024-09-05T01:15:18.877709+0000","last_undegraded":"2024-09-05T01:15:18.877709+0000","last_fullsized":"2024-09-05T01:15:18.877709+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:33.271142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876002+0000","last_change":"2024-09-05T01:13:04.189965+0000","last_active":"2024-09-05T01:15:18.876002+0000","last_peered":"2024-09-05T01:15:18.876002+0000","last_clean":"2024-09-05T01:15:18.876002+0000","last_became_active":"2024-09-05T01:13:04.189528+0000","last_became_peered":"2024-09-05T01:13:04.189528+0000","last_unstale":"2024-09-05T01:15:18.876002+0000","last_undegraded":"2024-09-05T01:15:18.876002+0000","last_fullsized":"2024-09-05T01:15:18.876002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:29:25.351305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872919+0000","last_change":"2024-09-05T01:13:12.920440+0000","last_active":"2024-09-05T01:15:18.872919+0000","last_peered":"2024-09-05T01:15:18.872919+0000","last_clean":"2024-09-05T01:15:18.872919+0000","last_became_active":"2024-09-05T01:13:12.920071+0000","last_became_peered":"2024-09-05T01:13:12.920071+0000","last_unstale":"2024-09-05T01:15:18.872919+0000","last_undegraded":"2024-09-05T01:15:18.872919+0000","last_fullsized":"2024-09-05T01:15:18.872919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:31:25.966790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877435+0000","last_change":"2024-09-05T01:13:05.900715+0000","last_active":"2024-09-05T01:15:18.877435+0000","last_peered":"2024-09-05T01:15:18.877435+0000","last_clean":"2024-09-05T01:15:18.877435+0000","last_became_active":"2024-09-05T01:13:05.900400+0000","last_became_peered":"2024-09-05T01:13:05.900400+0000","last_unstale":"2024-09-05T01:15:18.877435+0000","last_undegraded":"2024-09-05T01:15:18.877435+0000","last_fullsized":"2024-09-05T01:15:18.877435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:07.583838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879901+0000","last_change":"2024-09-05T01:13:03.883576+0000","last_active":"2024-09-05T01:15:18.879901+0000","last_peered":"2024-09-05T01:15:18.879901+0000","last_clean":"2024-09-05T01:15:18.879901+0000","last_became_active":"2024-09-05T01:13:03.883225+0000","last_became_peered":"2024-09-05T01:13:03.883225+0000","last_unstale":"2024-09-05T01:15:18.879901+0000","last_undegraded":"2024-09-05T01:15:18.879901+0000","last_fullsized":"2024-09-05T01:15:18.879901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:56:05.337105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892762+0000","last_change":"2024-09-05T01:13:12.919307+0000","last_active":"2024-09-05T01:15:18.892762+0000","last_peered":"2024-09-05T01:15:18.892762+0000","last_clean":"2024-09-05T01:15:18.892762+0000","last_became_active":"2024-09-05T01:13:12.918852+0000","last_became_peered":"2024-09-05T01:13:12.918852+0000","last_unstale":"2024-09-05T01:15:18.892762+0000","last_undegraded":"2024-09-05T01:15:18.892762+0000","last_fullsized":"2024-09-05T01:15:18.892762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:11:14.301507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877936+0000","last_change":"2024-09-05T01:13:05.894575+0000","last_active":"2024-09-05T01:15:18.877936+0000","last_peered":"2024-09-05T01:15:18.877936+0000","last_clean":"2024-09-05T01:15:18.877936+0000","last_became_active":"2024-09-05T01:13:05.894252+0000","last_became_peered":"2024-09-05T01:13:05.894252+0000","last_unstale":"2024-09-05T01:15:18.877936+0000","last_undegraded":"2024-09-05T01:15:18.877936+0000","last_fullsized":"2024-09-05T01:15:18.877936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:20:41.400441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879564+0000","last_change":"2024-09-05T01:15:18.879564+0000","last_active":"2024-09-05T01:15:18.879564+0000","last_peered":"2024-09-05T01:15:18.879564+0000","last_clean":"2024-09-05T01:15:18.879564+0000","last_became_active":"2024-09-05T01:15:18.876215+0000","last_became_peered":"2024-09-05T01:15:18.876215+0000","last_unstale":"2024-09-05T01:15:18.879564+0000","last_undegraded":"2024-09-05T01:15:18.879564+0000","last_fullsized":"2024-09-05T01:15:18.879564+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:54:11.502597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877679+0000","last_change":"2024-09-05T01:14:18.913738+0000","last_active":"2024-09-05T01:15:18.877679+0000","last_peered":"2024-09-05T01:15:18.877679+0000","last_clean":"2024-09-05T01:15:18.877679+0000","last_became_active":"2024-09-05T01:14:18.913084+0000","last_became_peered":"2024-09-05T01:14:18.913084+0000","last_unstale":"2024-09-05T01:15:18.877679+0000","last_undegraded":"2024-09-05T01:15:18.877679+0000","last_fullsized":"2024-09-05T01:15:18.877679+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:00:43.099911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879092+0000","last_change":"2024-09-05T01:13:05.894146+0000","last_active":"2024-09-05T01:15:18.879092+0000","last_peered":"2024-09-05T01:15:18.879092+0000","last_clean":"2024-09-05T01:15:18.879092+0000","last_became_active":"2024-09-05T01:13:05.893684+0000","last_became_peered":"2024-09-05T01:13:05.893684+0000","last_unstale":"2024-09-05T01:15:18.879092+0000","last_undegraded":"2024-09-05T01:15:18.879092+0000","last_fullsized":"2024-09-05T01:15:18.879092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:23:46.726844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.880937+0000","last_change":"2024-09-05T01:13:03.877305+0000","last_active":"2024-09-05T01:15:18.880937+0000","last_peered":"2024-09-05T01:15:18.880937+0000","last_clean":"2024-09-05T01:15:18.880937+0000","last_became_active":"2024-09-05T01:13:03.876906+0000","last_became_peered":"2024-09-05T01:13:03.876906+0000","last_unstale":"2024-09-05T01:15:18.880937+0000","last_undegraded":"2024-09-05T01:15:18.880937+0000","last_fullsized":"2024-09-05T01:15:18.880937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:56:56.724668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876614+0000","last_change":"2024-09-05T01:13:12.916334+0000","last_active":"2024-09-05T01:15:18.876614+0000","last_peered":"2024-09-05T01:15:18.876614+0000","last_clean":"2024-09-05T01:15:18.876614+0000","last_became_active":"2024-09-05T01:13:12.915916+0000","last_became_peered":"2024-09-05T01:13:12.915916+0000","last_unstale":"2024-09-05T01:15:18.876614+0000","last_undegraded":"2024-09-05T01:15:18.876614+0000","last_fullsized":"2024-09-05T01:15:18.876614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:55:54.298136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876672+0000","last_change":"2024-09-05T01:13:05.900914+0000","last_active":"2024-09-05T01:15:18.876672+0000","last_peered":"2024-09-05T01:15:18.876672+0000","last_clean":"2024-09-05T01:15:18.876672+0000","last_became_active":"2024-09-05T01:13:05.900587+0000","last_became_peered":"2024-09-05T01:13:05.900587+0000","last_unstale":"2024-09-05T01:15:18.876672+0000","last_undegraded":"2024-09-05T01:15:18.876672+0000","last_fullsized":"2024-09-05T01:15:18.876672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:11:38.373900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878035+0000","last_change":"2024-09-05T01:13:03.966829+0000","last_active":"2024-09-05T01:15:18.878035+0000","last_peered":"2024-09-05T01:15:18.878035+0000","last_clean":"2024-09-05T01:15:18.878035+0000","last_became_active":"2024-09-05T01:13:03.966262+0000","last_became_peered":"2024-09-05T01:13:03.966262+0000","last_unstale":"2024-09-05T01:15:18.878035+0000","last_undegraded":"2024-09-05T01:15:18.878035+0000","last_fullsized":"2024-09-05T01:15:18.878035+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:42:22.920251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872466+0000","last_change":"2024-09-05T01:13:12.915212+0000","last_active":"2024-09-05T01:15:18.872466+0000","last_peered":"2024-09-05T01:15:18.872466+0000","last_clean":"2024-09-05T01:15:18.872466+0000","last_became_active":"2024-09-05T01:13:12.914853+0000","last_became_peered":"2024-09-05T01:13:12.914853+0000","last_unstale":"2024-09-05T01:15:18.872466+0000","last_undegraded":"2024-09-05T01:15:18.872466+0000","last_fullsized":"2024-09-05T01:15:18.872466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:30:46.090035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877066+0000","last_change":"2024-09-05T01:13:05.901222+0000","last_active":"2024-09-05T01:15:18.877066+0000","last_peered":"2024-09-05T01:15:18.877066+0000","last_clean":"2024-09-05T01:15:18.877066+0000","last_became_active":"2024-09-05T01:13:05.900333+0000","last_became_peered":"2024-09-05T01:13:05.900333+0000","last_unstale":"2024-09-05T01:15:18.877066+0000","last_undegraded":"2024-09-05T01:15:18.877066+0000","last_fullsized":"2024-09-05T01:15:18.877066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:34:33.941827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":144,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:25.997792+0000","last_change":"2024-09-05T01:13:03.883495+0000","last_active":"2024-09-05T01:15:25.997792+0000","last_peered":"2024-09-05T01:15:25.997792+0000","last_clean":"2024-09-05T01:15:25.997792+0000","last_became_active":"2024-09-05T01:13:03.883173+0000","last_became_peered":"2024-09-05T01:13:03.883173+0000","last_unstale":"2024-09-05T01:15:25.997792+0000","last_undegraded":"2024-09-05T01:15:25.997792+0000","last_fullsized":"2024-09-05T01:15:25.997792+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:48:33.023018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1067,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876080+0000","last_change":"2024-09-05T01:13:12.916483+0000","last_active":"2024-09-05T01:15:18.876080+0000","last_peered":"2024-09-05T01:15:18.876080+0000","last_clean":"2024-09-05T01:15:18.876080+0000","last_became_active":"2024-09-05T01:13:12.916139+0000","last_became_peered":"2024-09-05T01:13:12.916139+0000","last_unstale":"2024-09-05T01:15:18.876080+0000","last_undegraded":"2024-09-05T01:15:18.876080+0000","last_fullsized":"2024-09-05T01:15:18.876080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:51:23.012081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877281+0000","last_change":"2024-09-05T01:13:05.890469+0000","last_active":"2024-09-05T01:15:18.877281+0000","last_peered":"2024-09-05T01:15:18.877281+0000","last_clean":"2024-09-05T01:15:18.877281+0000","last_became_active":"2024-09-05T01:13:05.890150+0000","last_became_peered":"2024-09-05T01:13:05.890150+0000","last_unstale":"2024-09-05T01:15:18.877281+0000","last_undegraded":"2024-09-05T01:15:18.877281+0000","last_fullsized":"2024-09-05T01:15:18.877281+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:23:41.429091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877183+0000","last_change":"2024-09-05T01:13:03.881318+0000","last_active":"2024-09-05T01:15:18.877183+0000","last_peered":"2024-09-05T01:15:18.877183+0000","last_clean":"2024-09-05T01:15:18.877183+0000","last_became_active":"2024-09-05T01:13:03.880976+0000","last_became_peered":"2024-09-05T01:13:03.880976+0000","last_unstale":"2024-09-05T01:15:18.877183+0000","last_undegraded":"2024-09-05T01:15:18.877183+0000","last_fullsized":"2024-09-05T01:15:18.877183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:45:36.581423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877256+0000","last_change":"2024-09-05T01:13:12.923900+0000","last_active":"2024-09-05T01:15:18.877256+0000","last_peered":"2024-09-05T01:15:18.877256+0000","last_clean":"2024-09-05T01:15:18.877256+0000","last_became_active":"2024-09-05T01:13:12.923495+0000","last_became_peered":"2024-09-05T01:13:12.923495+0000","last_unstale":"2024-09-05T01:15:18.877256+0000","last_undegraded":"2024-09-05T01:15:18.877256+0000","last_fullsized":"2024-09-05T01:15:18.877256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:26:04.690041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872199+0000","last_change":"2024-09-05T01:13:05.888608+0000","last_active":"2024-09-05T01:15:18.872199+0000","last_peered":"2024-09-05T01:15:18.872199+0000","last_clean":"2024-09-05T01:15:18.872199+0000","last_became_active":"2024-09-05T01:13:05.888295+0000","last_became_peered":"2024-09-05T01:13:05.888295+0000","last_unstale":"2024-09-05T01:15:18.872199+0000","last_undegraded":"2024-09-05T01:15:18.872199+0000","last_fullsized":"2024-09-05T01:15:18.872199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:22:49.962207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875174+0000","last_change":"2024-09-05T01:13:03.884894+0000","last_active":"2024-09-05T01:15:18.875174+0000","last_peered":"2024-09-05T01:15:18.875174+0000","last_clean":"2024-09-05T01:15:18.875174+0000","last_became_active":"2024-09-05T01:13:03.884513+0000","last_became_peered":"2024-09-05T01:13:03.884513+0000","last_unstale":"2024-09-05T01:15:18.875174+0000","last_undegraded":"2024-09-05T01:15:18.875174+0000","last_fullsized":"2024-09-05T01:15:18.875174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:44:02.917459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879784+0000","last_change":"2024-09-05T01:13:12.919867+0000","last_active":"2024-09-05T01:15:18.879784+0000","last_peered":"2024-09-05T01:15:18.879784+0000","last_clean":"2024-09-05T01:15:18.879784+0000","last_became_active":"2024-09-05T01:13:12.919434+0000","last_became_peered":"2024-09-05T01:13:12.919434+0000","last_unstale":"2024-09-05T01:15:18.879784+0000","last_undegraded":"2024-09-05T01:15:18.879784+0000","last_fullsized":"2024-09-05T01:15:18.879784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:11:14.405856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877835+0000","last_change":"2024-09-05T01:13:05.893565+0000","last_active":"2024-09-05T01:15:18.877835+0000","last_peered":"2024-09-05T01:15:18.877835+0000","last_clean":"2024-09-05T01:15:18.877835+0000","last_became_active":"2024-09-05T01:13:05.893050+0000","last_became_peered":"2024-09-05T01:13:05.893050+0000","last_unstale":"2024-09-05T01:15:18.877835+0000","last_undegraded":"2024-09-05T01:15:18.877835+0000","last_fullsized":"2024-09-05T01:15:18.877835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:24:08.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":287,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:21.904041+0000","last_change":"2024-09-05T01:15:18.902797+0000","last_active":"2024-09-05T01:15:21.904041+0000","last_peered":"2024-09-05T01:15:21.904041+0000","last_clean":"2024-09-05T01:15:21.904041+0000","last_became_active":"2024-09-05T01:15:18.896064+0000","last_became_peered":"2024-09-05T01:15:18.896064+0000","last_unstale":"2024-09-05T01:15:21.904041+0000","last_undegraded":"2024-09-05T01:15:21.904041+0000","last_fullsized":"2024-09-05T01:15:21.904041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:04:53.806916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62325,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":196,"num_read_kb":159,"num_write":36,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":62325,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874983+0000","last_change":"2024-09-05T01:14:18.840367+0000","last_active":"2024-09-05T01:15:18.874983+0000","last_peered":"2024-09-05T01:15:18.874983+0000","last_clean":"2024-09-05T01:15:18.874983+0000","last_became_active":"2024-09-05T01:14:18.839937+0000","last_became_peered":"2024-09-05T01:14:18.839937+0000","last_unstale":"2024-09-05T01:15:18.874983+0000","last_undegraded":"2024-09-05T01:15:18.874983+0000","last_fullsized":"2024-09-05T01:15:18.874983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:49:14.286631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873258+0000","last_change":"2024-09-05T01:13:05.882671+0000","last_active":"2024-09-05T01:15:18.873258+0000","last_peered":"2024-09-05T01:15:18.873258+0000","last_clean":"2024-09-05T01:15:18.873258+0000","last_became_active":"2024-09-05T01:13:05.882268+0000","last_became_peered":"2024-09-05T01:13:05.882268+0000","last_unstale":"2024-09-05T01:15:18.873258+0000","last_undegraded":"2024-09-05T01:15:18.873258+0000","last_fullsized":"2024-09-05T01:15:18.873258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:34:38.029418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879574+0000","last_change":"2024-09-05T01:13:03.966546+0000","last_active":"2024-09-05T01:15:18.879574+0000","last_peered":"2024-09-05T01:15:18.879574+0000","last_clean":"2024-09-05T01:15:18.879574+0000","last_became_active":"2024-09-05T01:13:03.966188+0000","last_became_peered":"2024-09-05T01:13:03.966188+0000","last_unstale":"2024-09-05T01:15:18.879574+0000","last_undegraded":"2024-09-05T01:15:18.879574+0000","last_fullsized":"2024-09-05T01:15:18.879574+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:07:46.979527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894131+0000","last_change":"2024-09-05T01:14:18.913011+0000","last_active":"2024-09-05T01:15:18.894131+0000","last_peered":"2024-09-05T01:15:18.894131+0000","last_clean":"2024-09-05T01:15:18.894131+0000","last_became_active":"2024-09-05T01:14:18.912546+0000","last_became_peered":"2024-09-05T01:14:18.912546+0000","last_unstale":"2024-09-05T01:15:18.894131+0000","last_undegraded":"2024-09-05T01:15:18.894131+0000","last_fullsized":"2024-09-05T01:15:18.894131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:11:02.202754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.893072+0000","last_change":"2024-09-05T01:13:05.888198+0000","last_active":"2024-09-05T01:15:18.893072+0000","last_peered":"2024-09-05T01:15:18.893072+0000","last_clean":"2024-09-05T01:15:18.893072+0000","last_became_active":"2024-09-05T01:13:05.887926+0000","last_became_peered":"2024-09-05T01:13:05.887926+0000","last_unstale":"2024-09-05T01:15:18.893072+0000","last_undegraded":"2024-09-05T01:15:18.893072+0000","last_fullsized":"2024-09-05T01:15:18.893072+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:57:58.199051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877804+0000","last_change":"2024-09-05T01:13:03.881820+0000","last_active":"2024-09-05T01:15:18.877804+0000","last_peered":"2024-09-05T01:15:18.877804+0000","last_clean":"2024-09-05T01:15:18.877804+0000","last_became_active":"2024-09-05T01:13:03.881460+0000","last_became_peered":"2024-09-05T01:13:03.881460+0000","last_unstale":"2024-09-05T01:15:18.877804+0000","last_undegraded":"2024-09-05T01:15:18.877804+0000","last_fullsized":"2024-09-05T01:15:18.877804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:20:31.189442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875496+0000","last_change":"2024-09-05T01:13:12.922759+0000","last_active":"2024-09-05T01:15:18.875496+0000","last_peered":"2024-09-05T01:15:18.875496+0000","last_clean":"2024-09-05T01:15:18.875496+0000","last_became_active":"2024-09-05T01:13:12.922136+0000","last_became_peered":"2024-09-05T01:13:12.922136+0000","last_unstale":"2024-09-05T01:15:18.875496+0000","last_undegraded":"2024-09-05T01:15:18.875496+0000","last_fullsized":"2024-09-05T01:15:18.875496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:29:22.364022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873793+0000","last_change":"2024-09-05T01:13:05.889973+0000","last_active":"2024-09-05T01:15:18.873793+0000","last_peered":"2024-09-05T01:15:18.873793+0000","last_clean":"2024-09-05T01:15:18.873793+0000","last_became_active":"2024-09-05T01:13:05.889566+0000","last_became_peered":"2024-09-05T01:13:05.889566+0000","last_unstale":"2024-09-05T01:15:18.873793+0000","last_undegraded":"2024-09-05T01:15:18.873793+0000","last_fullsized":"2024-09-05T01:15:18.873793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:04:17.709348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:20.892189+0000","last_change":"2024-09-05T01:13:03.881176+0000","last_active":"2024-09-05T01:15:20.892189+0000","last_peered":"2024-09-05T01:15:20.892189+0000","last_clean":"2024-09-05T01:15:20.892189+0000","last_became_active":"2024-09-05T01:13:03.880736+0000","last_became_peered":"2024-09-05T01:13:03.880736+0000","last_unstale":"2024-09-05T01:15:20.892189+0000","last_undegraded":"2024-09-05T01:15:20.892189+0000","last_fullsized":"2024-09-05T01:15:20.892189+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:33:11.486646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":93,"num_read_kb":93,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874862+0000","last_change":"2024-09-05T01:13:12.920256+0000","last_active":"2024-09-05T01:15:18.874862+0000","last_peered":"2024-09-05T01:15:18.874862+0000","last_clean":"2024-09-05T01:15:18.874862+0000","last_became_active":"2024-09-05T01:13:12.915075+0000","last_became_peered":"2024-09-05T01:13:12.915075+0000","last_unstale":"2024-09-05T01:15:18.874862+0000","last_undegraded":"2024-09-05T01:15:18.874862+0000","last_fullsized":"2024-09-05T01:15:18.874862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:05:12.561131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892843+0000","last_change":"2024-09-05T01:13:12.915415+0000","last_active":"2024-09-05T01:15:18.892843+0000","last_peered":"2024-09-05T01:15:18.892843+0000","last_clean":"2024-09-05T01:15:18.892843+0000","last_became_active":"2024-09-05T01:13:12.914831+0000","last_became_peered":"2024-09-05T01:13:12.914831+0000","last_unstale":"2024-09-05T01:15:18.892843+0000","last_undegraded":"2024-09-05T01:15:18.892843+0000","last_fullsized":"2024-09-05T01:15:18.892843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:35:54.130784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875596+0000","last_change":"2024-09-05T01:13:05.883934+0000","last_active":"2024-09-05T01:15:18.875596+0000","last_peered":"2024-09-05T01:15:18.875596+0000","last_clean":"2024-09-05T01:15:18.875596+0000","last_became_active":"2024-09-05T01:13:05.883471+0000","last_became_peered":"2024-09-05T01:13:05.883471+0000","last_unstale":"2024-09-05T01:15:18.875596+0000","last_undegraded":"2024-09-05T01:15:18.875596+0000","last_fullsized":"2024-09-05T01:15:18.875596+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:35:05.515556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876372+0000","last_change":"2024-09-05T01:13:03.881677+0000","last_active":"2024-09-05T01:15:18.876372+0000","last_peered":"2024-09-05T01:15:18.876372+0000","last_clean":"2024-09-05T01:15:18.876372+0000","last_became_active":"2024-09-05T01:13:03.880926+0000","last_became_peered":"2024-09-05T01:13:03.880926+0000","last_unstale":"2024-09-05T01:15:18.876372+0000","last_undegraded":"2024-09-05T01:15:18.876372+0000","last_fullsized":"2024-09-05T01:15:18.876372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:41:30.938041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879805+0000","last_change":"2024-09-05T01:13:12.917638+0000","last_active":"2024-09-05T01:15:18.879805+0000","last_peered":"2024-09-05T01:15:18.879805+0000","last_clean":"2024-09-05T01:15:18.879805+0000","last_became_active":"2024-09-05T01:13:12.917209+0000","last_became_peered":"2024-09-05T01:13:12.917209+0000","last_unstale":"2024-09-05T01:15:18.879805+0000","last_undegraded":"2024-09-05T01:15:18.879805+0000","last_fullsized":"2024-09-05T01:15:18.879805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:39:06.684225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874509+0000","last_change":"2024-09-05T01:13:05.891650+0000","last_active":"2024-09-05T01:15:18.874509+0000","last_peered":"2024-09-05T01:15:18.874509+0000","last_clean":"2024-09-05T01:15:18.874509+0000","last_became_active":"2024-09-05T01:13:05.891242+0000","last_became_peered":"2024-09-05T01:13:05.891242+0000","last_unstale":"2024-09-05T01:15:18.874509+0000","last_undegraded":"2024-09-05T01:15:18.874509+0000","last_fullsized":"2024-09-05T01:15:18.874509+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:36:32.793355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872057+0000","last_change":"2024-09-05T01:13:03.876038+0000","last_active":"2024-09-05T01:15:18.872057+0000","last_peered":"2024-09-05T01:15:18.872057+0000","last_clean":"2024-09-05T01:15:18.872057+0000","last_became_active":"2024-09-05T01:13:03.875485+0000","last_became_peered":"2024-09-05T01:13:03.875485+0000","last_unstale":"2024-09-05T01:15:18.872057+0000","last_undegraded":"2024-09-05T01:15:18.872057+0000","last_fullsized":"2024-09-05T01:15:18.872057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:35:48.524559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875691+0000","last_change":"2024-09-05T01:13:03.881766+0000","last_active":"2024-09-05T01:15:18.875691+0000","last_peered":"2024-09-05T01:15:18.875691+0000","last_clean":"2024-09-05T01:15:18.875691+0000","last_became_active":"2024-09-05T01:13:03.881470+0000","last_became_peered":"2024-09-05T01:13:03.881470+0000","last_unstale":"2024-09-05T01:15:18.875691+0000","last_undegraded":"2024-09-05T01:15:18.875691+0000","last_fullsized":"2024-09-05T01:15:18.875691+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:19:03.871006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875643+0000","last_change":"2024-09-05T01:14:18.912517+0000","last_active":"2024-09-05T01:15:18.875643+0000","last_peered":"2024-09-05T01:15:18.875643+0000","last_clean":"2024-09-05T01:15:18.875643+0000","last_became_active":"2024-09-05T01:14:18.912074+0000","last_became_peered":"2024-09-05T01:14:18.912074+0000","last_unstale":"2024-09-05T01:15:18.875643+0000","last_undegraded":"2024-09-05T01:15:18.875643+0000","last_fullsized":"2024-09-05T01:15:18.875643+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:20:16.172775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874931+0000","last_change":"2024-09-05T01:13:12.927680+0000","last_active":"2024-09-05T01:15:18.874931+0000","last_peered":"2024-09-05T01:15:18.874931+0000","last_clean":"2024-09-05T01:15:18.874931+0000","last_became_active":"2024-09-05T01:13:12.927341+0000","last_became_peered":"2024-09-05T01:13:12.927341+0000","last_unstale":"2024-09-05T01:15:18.874931+0000","last_undegraded":"2024-09-05T01:15:18.874931+0000","last_fullsized":"2024-09-05T01:15:18.874931+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:28:24.950217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'8","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:23.898003+0000","last_change":"2024-09-05T01:15:18.891283+0000","last_active":"2024-09-05T01:15:23.898003+0000","last_peered":"2024-09-05T01:15:23.898003+0000","last_clean":"2024-09-05T01:15:23.898003+0000","last_became_active":"2024-09-05T01:15:18.882289+0000","last_became_peered":"2024-09-05T01:15:18.882289+0000","last_unstale":"2024-09-05T01:15:23.898003+0000","last_undegraded":"2024-09-05T01:15:23.898003+0000","last_fullsized":"2024-09-05T01:15:23.898003+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:02.863670+0000","last_clean_scrub_stamp":"2024-09-05T01:13:02.863670+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:45:12.506629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1635,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":95,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1156,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878425+0000","last_change":"2024-09-05T01:13:05.886407+0000","last_active":"2024-09-05T01:15:18.878425+0000","last_peered":"2024-09-05T01:15:18.878425+0000","last_clean":"2024-09-05T01:15:18.878425+0000","last_became_active":"2024-09-05T01:13:05.886063+0000","last_became_peered":"2024-09-05T01:13:05.886063+0000","last_unstale":"2024-09-05T01:15:18.878425+0000","last_undegraded":"2024-09-05T01:15:18.878425+0000","last_fullsized":"2024-09-05T01:15:18.878425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:40.112032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875180+0000","last_change":"2024-09-05T01:13:12.924315+0000","last_active":"2024-09-05T01:15:18.875180+0000","last_peered":"2024-09-05T01:15:18.875180+0000","last_clean":"2024-09-05T01:15:18.875180+0000","last_became_active":"2024-09-05T01:13:12.923501+0000","last_became_peered":"2024-09-05T01:13:12.923501+0000","last_unstale":"2024-09-05T01:15:18.875180+0000","last_undegraded":"2024-09-05T01:15:18.875180+0000","last_fullsized":"2024-09-05T01:15:18.875180+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:11.898755+0000","last_clean_scrub_stamp":"2024-09-05T01:13:11.898755+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:54:38.430615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877663+0000","last_change":"2024-09-05T01:13:05.901490+0000","last_active":"2024-09-05T01:15:18.877663+0000","last_peered":"2024-09-05T01:15:18.877663+0000","last_clean":"2024-09-05T01:15:18.877663+0000","last_became_active":"2024-09-05T01:13:05.901116+0000","last_became_peered":"2024-09-05T01:13:05.901116+0000","last_unstale":"2024-09-05T01:15:18.877663+0000","last_undegraded":"2024-09-05T01:15:18.877663+0000","last_fullsized":"2024-09-05T01:15:18.877663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:25:29.685037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875455+0000","last_change":"2024-09-05T01:13:05.894681+0000","last_active":"2024-09-05T01:15:18.875455+0000","last_peered":"2024-09-05T01:15:18.875455+0000","last_clean":"2024-09-05T01:15:18.875455+0000","last_became_active":"2024-09-05T01:13:05.892730+0000","last_became_peered":"2024-09-05T01:13:05.892730+0000","last_unstale":"2024-09-05T01:15:18.875455+0000","last_undegraded":"2024-09-05T01:15:18.875455+0000","last_fullsized":"2024-09-05T01:15:18.875455+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:00:49.426945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874866+0000","last_change":"2024-09-05T01:13:05.898147+0000","last_active":"2024-09-05T01:15:18.874866+0000","last_peered":"2024-09-05T01:15:18.874866+0000","last_clean":"2024-09-05T01:15:18.874866+0000","last_became_active":"2024-09-05T01:13:05.897806+0000","last_became_peered":"2024-09-05T01:13:05.897806+0000","last_unstale":"2024-09-05T01:15:18.874866+0000","last_undegraded":"2024-09-05T01:15:18.874866+0000","last_fullsized":"2024-09-05T01:15:18.874866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:11:26.338712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892838+0000","last_change":"2024-09-05T01:13:05.883446+0000","last_active":"2024-09-05T01:15:18.892838+0000","last_peered":"2024-09-05T01:15:18.892838+0000","last_clean":"2024-09-05T01:15:18.892838+0000","last_became_active":"2024-09-05T01:13:05.883161+0000","last_became_peered":"2024-09-05T01:13:05.883161+0000","last_unstale":"2024-09-05T01:15:18.892838+0000","last_undegraded":"2024-09-05T01:15:18.892838+0000","last_fullsized":"2024-09-05T01:15:18.892838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:15:01.687764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872553+0000","last_change":"2024-09-05T01:13:05.884971+0000","last_active":"2024-09-05T01:15:18.872553+0000","last_peered":"2024-09-05T01:15:18.872553+0000","last_clean":"2024-09-05T01:15:18.872553+0000","last_became_active":"2024-09-05T01:13:05.884653+0000","last_became_peered":"2024-09-05T01:13:05.884653+0000","last_unstale":"2024-09-05T01:15:18.872553+0000","last_undegraded":"2024-09-05T01:15:18.872553+0000","last_fullsized":"2024-09-05T01:15:18.872553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:45:29.344729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.878714+0000","last_change":"2024-09-05T01:13:05.898528+0000","last_active":"2024-09-05T01:15:18.878714+0000","last_peered":"2024-09-05T01:15:18.878714+0000","last_clean":"2024-09-05T01:15:18.878714+0000","last_became_active":"2024-09-05T01:13:05.898214+0000","last_became_peered":"2024-09-05T01:13:05.898214+0000","last_unstale":"2024-09-05T01:15:18.878714+0000","last_undegraded":"2024-09-05T01:15:18.878714+0000","last_fullsized":"2024-09-05T01:15:18.878714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:54:16.912556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876503+0000","last_change":"2024-09-05T01:13:05.901673+0000","last_active":"2024-09-05T01:15:18.876503+0000","last_peered":"2024-09-05T01:15:18.876503+0000","last_clean":"2024-09-05T01:15:18.876503+0000","last_became_active":"2024-09-05T01:13:05.901319+0000","last_became_peered":"2024-09-05T01:13:05.901319+0000","last_unstale":"2024-09-05T01:15:18.876503+0000","last_undegraded":"2024-09-05T01:15:18.876503+0000","last_fullsized":"2024-09-05T01:15:18.876503+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:53:57.367234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875119+0000","last_change":"2024-09-05T01:13:05.893969+0000","last_active":"2024-09-05T01:15:18.875119+0000","last_peered":"2024-09-05T01:15:18.875119+0000","last_clean":"2024-09-05T01:15:18.875119+0000","last_became_active":"2024-09-05T01:13:05.893645+0000","last_became_peered":"2024-09-05T01:13:05.893645+0000","last_unstale":"2024-09-05T01:15:18.875119+0000","last_undegraded":"2024-09-05T01:15:18.875119+0000","last_fullsized":"2024-09-05T01:15:18.875119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:32:45.126333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875850+0000","last_change":"2024-09-05T01:13:05.898158+0000","last_active":"2024-09-05T01:15:18.875850+0000","last_peered":"2024-09-05T01:15:18.875850+0000","last_clean":"2024-09-05T01:15:18.875850+0000","last_became_active":"2024-09-05T01:13:05.897866+0000","last_became_peered":"2024-09-05T01:13:05.897866+0000","last_unstale":"2024-09-05T01:15:18.875850+0000","last_undegraded":"2024-09-05T01:15:18.875850+0000","last_fullsized":"2024-09-05T01:15:18.875850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:48:24.631280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874525+0000","last_change":"2024-09-05T01:13:05.891711+0000","last_active":"2024-09-05T01:15:18.874525+0000","last_peered":"2024-09-05T01:15:18.874525+0000","last_clean":"2024-09-05T01:15:18.874525+0000","last_became_active":"2024-09-05T01:13:05.891226+0000","last_became_peered":"2024-09-05T01:13:05.891226+0000","last_unstale":"2024-09-05T01:15:18.874525+0000","last_undegraded":"2024-09-05T01:15:18.874525+0000","last_fullsized":"2024-09-05T01:15:18.874525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:28:12.774852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873202+0000","last_change":"2024-09-05T01:13:05.890729+0000","last_active":"2024-09-05T01:15:18.873202+0000","last_peered":"2024-09-05T01:15:18.873202+0000","last_clean":"2024-09-05T01:15:18.873202+0000","last_became_active":"2024-09-05T01:13:05.888763+0000","last_became_peered":"2024-09-05T01:13:05.888763+0000","last_unstale":"2024-09-05T01:15:18.873202+0000","last_undegraded":"2024-09-05T01:15:18.873202+0000","last_fullsized":"2024-09-05T01:15:18.873202+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:17:36.002368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872496+0000","last_change":"2024-09-05T01:13:05.885980+0000","last_active":"2024-09-05T01:15:18.872496+0000","last_peered":"2024-09-05T01:15:18.872496+0000","last_clean":"2024-09-05T01:15:18.872496+0000","last_became_active":"2024-09-05T01:13:05.885652+0000","last_became_peered":"2024-09-05T01:13:05.885652+0000","last_unstale":"2024-09-05T01:15:18.872496+0000","last_undegraded":"2024-09-05T01:15:18.872496+0000","last_fullsized":"2024-09-05T01:15:18.872496+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:09:18.031048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877437+0000","last_change":"2024-09-05T01:14:18.840355+0000","last_active":"2024-09-05T01:15:18.877437+0000","last_peered":"2024-09-05T01:15:18.877437+0000","last_clean":"2024-09-05T01:15:18.877437+0000","last_became_active":"2024-09-05T01:14:18.839887+0000","last_became_peered":"2024-09-05T01:14:18.839887+0000","last_unstale":"2024-09-05T01:15:18.877437+0000","last_undegraded":"2024-09-05T01:15:18.877437+0000","last_fullsized":"2024-09-05T01:15:18.877437+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:50:49.152575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875539+0000","last_change":"2024-09-05T01:13:05.896255+0000","last_active":"2024-09-05T01:15:18.875539+0000","last_peered":"2024-09-05T01:15:18.875539+0000","last_clean":"2024-09-05T01:15:18.875539+0000","last_became_active":"2024-09-05T01:13:05.895917+0000","last_became_peered":"2024-09-05T01:13:05.895917+0000","last_unstale":"2024-09-05T01:15:18.875539+0000","last_undegraded":"2024-09-05T01:15:18.875539+0000","last_fullsized":"2024-09-05T01:15:18.875539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:55:53.035406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.876230+0000","last_change":"2024-09-05T01:13:05.895913+0000","last_active":"2024-09-05T01:15:18.876230+0000","last_peered":"2024-09-05T01:15:18.876230+0000","last_clean":"2024-09-05T01:15:18.876230+0000","last_became_active":"2024-09-05T01:13:05.895579+0000","last_became_peered":"2024-09-05T01:13:05.895579+0000","last_unstale":"2024-09-05T01:15:18.876230+0000","last_undegraded":"2024-09-05T01:15:18.876230+0000","last_fullsized":"2024-09-05T01:15:18.876230+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T07:46:41.618910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879606+0000","last_change":"2024-09-05T01:13:05.896975+0000","last_active":"2024-09-05T01:15:18.879606+0000","last_peered":"2024-09-05T01:15:18.879606+0000","last_clean":"2024-09-05T01:15:18.879606+0000","last_became_active":"2024-09-05T01:13:05.896603+0000","last_became_peered":"2024-09-05T01:13:05.896603+0000","last_unstale":"2024-09-05T01:15:18.879606+0000","last_undegraded":"2024-09-05T01:15:18.879606+0000","last_fullsized":"2024-09-05T01:15:18.879606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:33:16.490805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872781+0000","last_change":"2024-09-05T01:13:05.891107+0000","last_active":"2024-09-05T01:15:18.872781+0000","last_peered":"2024-09-05T01:15:18.872781+0000","last_clean":"2024-09-05T01:15:18.872781+0000","last_became_active":"2024-09-05T01:13:05.888293+0000","last_became_peered":"2024-09-05T01:13:05.888293+0000","last_unstale":"2024-09-05T01:15:18.872781+0000","last_undegraded":"2024-09-05T01:15:18.872781+0000","last_fullsized":"2024-09-05T01:15:18.872781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T09:32:06.358234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892640+0000","last_change":"2024-09-05T01:13:05.890812+0000","last_active":"2024-09-05T01:15:18.892640+0000","last_peered":"2024-09-05T01:15:18.892640+0000","last_clean":"2024-09-05T01:15:18.892640+0000","last_became_active":"2024-09-05T01:13:05.889754+0000","last_became_peered":"2024-09-05T01:13:05.889754+0000","last_unstale":"2024-09-05T01:15:18.892640+0000","last_undegraded":"2024-09-05T01:15:18.892640+0000","last_fullsized":"2024-09-05T01:15:18.892640+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T03:23:39.310240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.884537+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.884230+0000","last_became_peered":"2024-09-05T01:13:05.884230+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:54:00.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.872317+0000","last_change":"2024-09-05T01:14:18.836841+0000","last_active":"2024-09-05T01:15:18.872317+0000","last_peered":"2024-09-05T01:15:18.872317+0000","last_clean":"2024-09-05T01:15:18.872317+0000","last_became_active":"2024-09-05T01:14:18.836464+0000","last_became_peered":"2024-09-05T01:14:18.836464+0000","last_unstale":"2024-09-05T01:15:18.872317+0000","last_undegraded":"2024-09-05T01:15:18.872317+0000","last_fullsized":"2024-09-05T01:15:18.872317+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:21:02.858269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894772+0000","last_change":"2024-09-05T01:13:05.889805+0000","last_active":"2024-09-05T01:15:18.894772+0000","last_peered":"2024-09-05T01:15:18.894772+0000","last_clean":"2024-09-05T01:15:18.894772+0000","last_became_active":"2024-09-05T01:13:05.888905+0000","last_became_peered":"2024-09-05T01:13:05.888905+0000","last_unstale":"2024-09-05T01:15:18.894772+0000","last_undegraded":"2024-09-05T01:15:18.894772+0000","last_fullsized":"2024-09-05T01:15:18.894772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T08:54:00.287611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877636+0000","last_change":"2024-09-05T01:13:05.898093+0000","last_active":"2024-09-05T01:15:18.877636+0000","last_peered":"2024-09-05T01:15:18.877636+0000","last_clean":"2024-09-05T01:15:18.877636+0000","last_became_active":"2024-09-05T01:13:05.897751+0000","last_became_peered":"2024-09-05T01:13:05.897751+0000","last_unstale":"2024-09-05T01:15:18.877636+0000","last_undegraded":"2024-09-05T01:15:18.877636+0000","last_fullsized":"2024-09-05T01:15:18.877636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:04:24.456125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875974+0000","last_change":"2024-09-05T01:13:05.888961+0000","last_active":"2024-09-05T01:15:18.875974+0000","last_peered":"2024-09-05T01:15:18.875974+0000","last_clean":"2024-09-05T01:15:18.875974+0000","last_became_active":"2024-09-05T01:13:05.888602+0000","last_became_peered":"2024-09-05T01:13:05.888602+0000","last_unstale":"2024-09-05T01:15:18.875974+0000","last_undegraded":"2024-09-05T01:15:18.875974+0000","last_fullsized":"2024-09-05T01:15:18.875974+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T06:30:49.088240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.873830+0000","last_change":"2024-09-05T01:13:05.897975+0000","last_active":"2024-09-05T01:15:18.873830+0000","last_peered":"2024-09-05T01:15:18.873830+0000","last_clean":"2024-09-05T01:15:18.873830+0000","last_became_active":"2024-09-05T01:13:05.896935+0000","last_became_peered":"2024-09-05T01:13:05.896935+0000","last_unstale":"2024-09-05T01:15:18.873830+0000","last_undegraded":"2024-09-05T01:15:18.873830+0000","last_fullsized":"2024-09-05T01:15:18.873830+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T01:45:22.831840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.874356+0000","last_change":"2024-09-05T01:13:05.892720+0000","last_active":"2024-09-05T01:15:18.874356+0000","last_peered":"2024-09-05T01:15:18.874356+0000","last_clean":"2024-09-05T01:15:18.874356+0000","last_became_active":"2024-09-05T01:13:05.892431+0000","last_became_peered":"2024-09-05T01:13:05.892431+0000","last_unstale":"2024-09-05T01:15:18.874356+0000","last_undegraded":"2024-09-05T01:15:18.874356+0000","last_fullsized":"2024-09-05T01:15:18.874356+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:16:30.267371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.877120+0000","last_change":"2024-09-05T01:13:05.901364+0000","last_active":"2024-09-05T01:15:18.877120+0000","last_peered":"2024-09-05T01:15:18.877120+0000","last_clean":"2024-09-05T01:15:18.877120+0000","last_became_active":"2024-09-05T01:13:05.900933+0000","last_became_peered":"2024-09-05T01:13:05.900933+0000","last_unstale":"2024-09-05T01:15:18.877120+0000","last_undegraded":"2024-09-05T01:15:18.877120+0000","last_fullsized":"2024-09-05T01:15:18.877120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:36:36.910980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.875187+0000","last_change":"2024-09-05T01:13:05.888512+0000","last_active":"2024-09-05T01:15:18.875187+0000","last_peered":"2024-09-05T01:15:18.875187+0000","last_clean":"2024-09-05T01:15:18.875187+0000","last_became_active":"2024-09-05T01:13:05.888176+0000","last_became_peered":"2024-09-05T01:13:05.888176+0000","last_unstale":"2024-09-05T01:15:18.875187+0000","last_undegraded":"2024-09-05T01:15:18.875187+0000","last_fullsized":"2024-09-05T01:15:18.875187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T05:26:21.592378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879887+0000","last_change":"2024-09-05T01:13:05.883418+0000","last_active":"2024-09-05T01:15:18.879887+0000","last_peered":"2024-09-05T01:15:18.879887+0000","last_clean":"2024-09-05T01:15:18.879887+0000","last_became_active":"2024-09-05T01:13:05.883091+0000","last_became_peered":"2024-09-05T01:13:05.883091+0000","last_unstale":"2024-09-05T01:15:18.879887+0000","last_undegraded":"2024-09-05T01:15:18.879887+0000","last_fullsized":"2024-09-05T01:15:18.879887+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T10:11:14.554652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.894875+0000","last_change":"2024-09-05T01:13:05.890881+0000","last_active":"2024-09-05T01:15:18.894875+0000","last_peered":"2024-09-05T01:15:18.894875+0000","last_clean":"2024-09-05T01:15:18.894875+0000","last_became_active":"2024-09-05T01:13:05.890281+0000","last_became_peered":"2024-09-05T01:13:05.890281+0000","last_unstale":"2024-09-05T01:15:18.894875+0000","last_undegraded":"2024-09-05T01:15:18.894875+0000","last_fullsized":"2024-09-05T01:15:18.894875+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:38.682234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892726+0000","last_change":"2024-09-05T01:13:05.897010+0000","last_active":"2024-09-05T01:15:18.892726+0000","last_peered":"2024-09-05T01:15:18.892726+0000","last_clean":"2024-09-05T01:15:18.892726+0000","last_became_active":"2024-09-05T01:13:05.896661+0000","last_became_peered":"2024-09-05T01:13:05.896661+0000","last_unstale":"2024-09-05T01:15:18.892726+0000","last_undegraded":"2024-09-05T01:15:18.892726+0000","last_fullsized":"2024-09-05T01:15:18.892726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T04:49:57.444819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.892807+0000","last_change":"2024-09-05T01:13:05.891233+0000","last_active":"2024-09-05T01:15:18.892807+0000","last_peered":"2024-09-05T01:15:18.892807+0000","last_clean":"2024-09-05T01:15:18.892807+0000","last_became_active":"2024-09-05T01:13:05.888917+0000","last_became_peered":"2024-09-05T01:13:05.888917+0000","last_unstale":"2024-09-05T01:15:18.892807+0000","last_undegraded":"2024-09-05T01:15:18.892807+0000","last_fullsized":"2024-09-05T01:15:18.892807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T02:51:42.952780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-05T01:15:18.879435+0000","last_change":"2024-09-05T01:13:05.896784+0000","last_active":"2024-09-05T01:15:18.879435+0000","last_peered":"2024-09-05T01:15:18.879435+0000","last_clean":"2024-09-05T01:15:18.879435+0000","last_became_active":"2024-09-05T01:13:05.896442+0000","last_became_peered":"2024-09-05T01:13:05.896442+0000","last_unstale":"2024-09-05T01:15:18.879435+0000","last_undegraded":"2024-09-05T01:15:18.879435+0000","last_fullsized":"2024-09-05T01:15:18.879435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T01:13:04.867496+0000","last_clean_scrub_stamp":"2024-09-05T01:13:04.867496+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T11:10:47.866975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67427,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":594,"num_read_kb":369,"num_write":176,"num_write_kb":123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":63481,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":434176,"data_stored":209508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150,"ondisk_log_size":150,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579946,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":516334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1724416,"data_stored":1164292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972403,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1052672,"data_stored":519031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201337,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1048576,"data_stored":514855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430270,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32788,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712364,"statfs":{"total":95995035648,"available":95961460736,"internally_reserved":0,"allocated":1642496,"data_stored":1111865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593795,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":510111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822727,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1642496,"data_stored":1105339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":585271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215186,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1040384,"data_stored":510305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36448,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708704,"statfs":{"total":95995035648,"available":95957712896,"internally_reserved":0,"allocated":1048576,"data_stored":511905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":572825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":511499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":63903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T01:15:30.207 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:30.454 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:30.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 -- 172.21.15.129:0/2184992917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68106f10 msgr2=0x7f5a6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:30.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 --2- 172.21.15.129:0/2184992917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68106f10 0x7f5a6810ea90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009ca0 tx=0x7f5a6002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 -- 172.21.15.129:0/2184992917 shutdown_connections 2024-09-05T01:15:30.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 --2- 172.21.15.129:0/2184992917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68106f10 0x7f5a6810ea90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 --2- 172.21.15.129:0/2184992917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68102790 0x7f5a681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 --2- 172.21.15.129:0/2184992917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68101de0 0x7f5a681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 -- 172.21.15.129:0/2184992917 >> 172.21.15.129:0/2184992917 conn(0x7f5a680fd8c0 msgr2=0x7f5a680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 -- 172.21.15.129:0/2184992917 shutdown_connections 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 -- 172.21.15.129:0/2184992917 wait complete. 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.925+0000 7f5a700f5640 1 Processor -- start 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 -- start start 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68102790 0x7f5a68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 0x7f5a68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:30.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68111e20 con 0x7f5a68101de0 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68111ca0 con 0x7f5a68106f10 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a700f5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5a68111fa0 con 0x7f5a68102790 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52244/0 (socket says 172.21.15.129:52244) 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 -- 172.21.15.129:0/3194590825 learned_addr learned my addr 172.21.15.129:0/3194590825 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6d690640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68102790 0x7f5a68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6e692640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 0x7f5a68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 -- 172.21.15.129:0/3194590825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68102790 msgr2=0x7f5a68074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68102790 0x7f5a68074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 -- 172.21.15.129:0/3194590825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 msgr2=0x7f5a68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:30.940 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 0x7f5a68074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 -- 172.21.15.129:0/3194590825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a681a67e0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6e692640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 0x7f5a68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.926+0000 7f5a6de91640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d770 tx=0x7f5a6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.927+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64017070 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.927+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a681a6ad0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.927+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a640054d0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.927+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004cf0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.927+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a681a6fc0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.928+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a681021c0 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.929+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5a64004e90 con 0x7f5a68101de0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.929+0000 7f5a5effd640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 0x7f5a540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.929+0000 7f5a6d690640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 0x7f5a540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:30.941 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.929+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f5a640cf100 con 0x7f5a68101de0 2024-09-05T01:15:30.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.930+0000 7f5a6d690640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 0x7f5a540a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a68075f80 tx=0x7f5a58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:30.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:30.933+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a6409baf0 con 0x7f5a68101de0 2024-09-05T01:15:31.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.134+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a6810a150 con 0x7f5a68101de0 2024-09-05T01:15:31.135 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:30 smithi129 ceph-mon[30979]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:31.135 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:30 smithi129 ceph-mon[30979]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 5.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:31.136 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.135+0000 7f5a5effd640 1 -- 172.21.15.129:0/3194590825 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f5a6409b3f0 con 0x7f5a68101de0 2024-09-05T01:15:31.136 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:31.136 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":26,"btime":"2024-09-05T01:15:24:939831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:24.411921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:15:31.136 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 26 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 msgr2=0x7f5a540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 0x7f5a540a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5a68075f80 tx=0x7f5a58005f90 comp rx=0 tx=0).stop 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 msgr2=0x7f5a680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d770 tx=0x7f5a6400dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 shutdown_connections 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5a540a2c10 0x7f5a540a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:31.138 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5a68106f10 0x7f5a68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:31.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5a68102790 0x7f5a68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:31.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 --2- 172.21.15.129:0/3194590825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a68101de0 0x7f5a680744f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:31.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 >> 172.21.15.129:0/3194590825 conn(0x7f5a680fd8c0 msgr2=0x7f5a680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:31.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 shutdown_connections 2024-09-05T01:15:31.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:31.137+0000 7f5a700f5640 1 -- 172.21.15.129:0/3194590825 wait complete. 2024-09-05T01:15:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:30 smithi183 ceph-mon[34589]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:31.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:30 smithi183 ceph-mon[34589]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 5.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:30 smithi194 ceph-mon[34941]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T01:15:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:30 smithi194 ceph-mon[34941]: pgmap v356: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 5.7 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:31.796 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:32.046 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:32.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:31 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3194590825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:32.166 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:31 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3194590825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:31 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3194590825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 -- 172.21.15.129:0/4107328224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 msgr2=0x7f57581031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 --2- 172.21.15.129:0/4107328224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f57581031c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5754009900 tx=0x7f575402a480 comp rx=0 tx=0).stop 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 -- 172.21.15.129:0/4107328224 shutdown_connections 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 --2- 172.21.15.129:0/4107328224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 0x7f575810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 --2- 172.21.15.129:0/4107328224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 0x7f575810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 --2- 172.21.15.129:0/4107328224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f57581031c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 -- 172.21.15.129:0/4107328224 >> 172.21.15.129:0/4107328224 conn(0x7f57580fc7a0 msgr2=0x7f57580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 -- 172.21.15.129:0/4107328224 shutdown_connections 2024-09-05T01:15:32.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.516+0000 7f575f848640 1 -- 172.21.15.129:0/4107328224 wait complete. 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 Processor -- start 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 -- start start 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 0x7f5758100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 0x7f57581014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5758111e20 con 0x7f5758102de0 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5758111ca0 con 0x7f5758103700 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575f848640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5758111fa0 con 0x7f575810c180 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575d5e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575d5e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38004/0 (socket says 172.21.15.129:38004) 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.517+0000 7f575d5e4640 1 -- 172.21.15.129:0/2220180949 learned_addr learned my addr 172.21.15.129:0/2220180949 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575dde5640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 0x7f57581014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:32.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575cde3640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 0x7f5758100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 -- 172.21.15.129:0/2220180949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 msgr2=0x7f57581014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 0x7f57581014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 -- 172.21.15.129:0/2220180949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 msgr2=0x7f5758100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 0x7f5758100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 -- 172.21.15.129:0/2220180949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57581a6830 con 0x7f5758102de0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575d5e4640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5754005940 tx=0x7f5754004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575403f030 con 0x7f5758102de0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5754003aa0 con 0x7f5758102de0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57581a6ac0 con 0x7f5758102de0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754035070 con 0x7f5758102de0 2024-09-05T01:15:32.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.518+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57581a6fa0 con 0x7f5758102de0 2024-09-05T01:15:32.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.519+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57581031c0 con 0x7f5758102de0 2024-09-05T01:15:32.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.520+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5754042030 con 0x7f5758102de0 2024-09-05T01:15:32.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.521+0000 7f574e7fc640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 0x7f57440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:32.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.521+0000 7f575cde3640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 0x7f57440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:32.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.521+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f57540efdd0 con 0x7f5758102de0 2024-09-05T01:15:32.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.522+0000 7f575cde3640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 0x7f57440a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5748005fd0 tx=0x7f5748005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:32.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.526+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57540bc7c0 con 0x7f5758102de0 2024-09-05T01:15:32.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.724+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5758106cd0 con 0x7f5758102de0 2024-09-05T01:15:32.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.725+0000 7f574e7fc640 1 -- 172.21.15.129:0/2220180949 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f57540bc0c0 con 0x7f5758102de0 2024-09-05T01:15:32.725 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:32.726 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":26,"btime":"2024-09-05T01:15:24:939831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:24.411921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:15:32.726 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 26 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 msgr2=0x7f57440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 0x7f57440a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5748005fd0 tx=0x7f5748005850 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 msgr2=0x7f5758100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5754005940 tx=0x7f5754004270 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 shutdown_connections 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57440a2c10 0x7f57440a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f575810c180 0x7f57581014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5758103700 0x7f5758100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 --2- 172.21.15.129:0/2220180949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5758102de0 0x7f5758100a40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 >> 172.21.15.129:0/2220180949 conn(0x7f57580fc7a0 msgr2=0x7f57580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 shutdown_connections 2024-09-05T01:15:32.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:32.727+0000 7f575f848640 1 -- 172.21.15.129:0/2220180949 wait complete. 2024-09-05T01:15:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:32 smithi129 ceph-mon[30979]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 5.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:32 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2220180949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:32 smithi183 ceph-mon[34589]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 5.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:32 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2220180949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:32 smithi194 ceph-mon[34941]: pgmap v357: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 5.2 KiB/s, 0 keys/s, 1 objects/s recovering 2024-09-05T01:15:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:32 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2220180949' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:33.412 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:15:33.656 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 -- 172.21.15.129:0/1041679126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c106f30 msgr2=0x7f546c10eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 --2- 172.21.15.129:0/1041679126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c106f30 0x7f546c10eab0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5464009ca0 tx=0x7f546402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 -- 172.21.15.129:0/1041679126 shutdown_connections 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 --2- 172.21.15.129:0/1041679126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c106f30 0x7f546c10eab0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 --2- 172.21.15.129:0/1041679126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c100cc0 0x7f546c106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 --2- 172.21.15.129:0/1041679126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 0x7f546c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 -- 172.21.15.129:0/1041679126 >> 172.21.15.129:0/1041679126 conn(0x7f546c077210 msgr2=0x7f546c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 -- 172.21.15.129:0/1041679126 shutdown_connections 2024-09-05T01:15:34.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.133+0000 7f54711be640 1 -- 172.21.15.129:0/1041679126 wait complete. 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.134+0000 7f54711be640 1 Processor -- start 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.134+0000 7f54711be640 1 -- start start 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.134+0000 7f54711be640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 0x7f546c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.134+0000 7f54711be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c106f30 0x7f546c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f546c111e40 con 0x7f546c100cc0 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f546c111cc0 con 0x7f546c100310 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f546c111fc0 con 0x7f546c106f30 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38010/0 (socket says 172.21.15.129:38010) 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 -- 172.21.15.129:0/148213721 learned_addr learned my addr 172.21.15.129:0/148213721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:34.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546b59e640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c106f30 0x7f546c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546ad9d640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 0x7f546c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 -- 172.21.15.129:0/148213721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c106f30 msgr2=0x7f546c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c106f30 0x7f546c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 -- 172.21.15.129:0/148213721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 msgr2=0x7f546c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 0x7f546c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 -- 172.21.15.129:0/148213721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c19d2d0 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f546a59c640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f546000d770 tx=0x7f546000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460017070 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c19d5c0 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54600054d0 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460004cf0 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.135+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c1a4ea0 con 0x7f546c100cc0 2024-09-05T01:15:34.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.136+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c1006f0 con 0x7f546c100cc0 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.137+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f5460004e90 con 0x7f546c100cc0 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.138+0000 7f5453fff640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 0x7f54540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.138+0000 7f546ad9d640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 0x7f54540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.138+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f54600cef00 con 0x7f546c100cc0 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.139+0000 7f546ad9d640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 0x7f54540a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f545c00bbe0 tx=0x7f545c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:34.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.142+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f546009b2a0 con 0x7f546c100cc0 2024-09-05T01:15:34.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.310+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f546c10a170 con 0x7f546c100cc0 2024-09-05T01:15:34.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.312+0000 7f5453fff640 1 -- 172.21.15.129:0/148213721 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+25009 (secure 0 0 0) 0x7f546009f5a0 con 0x7f546c100cc0 2024-09-05T01:15:34.313 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:34.313 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":90,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:15:18.868971+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 msgr2=0x7f54540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 0x7f54540a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f545c00bbe0 tx=0x7f545c00b390 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 msgr2=0x7f546c198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f546000d770 tx=0x7f546000dc30 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 shutdown_connections 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f54540a2c10 0x7f54540a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f546c106f30 0x7f546c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f546c100cc0 0x7f546c198860 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 --2- 172.21.15.129:0/148213721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f546c100310 0x7f546c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:34.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 >> 172.21.15.129:0/148213721 conn(0x7f546c077210 msgr2=0x7f546c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:34.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 shutdown_connections 2024-09-05T01:15:34.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:34.314+0000 7f54711be640 1 -- 172.21.15.129:0/148213721 wait complete. 2024-09-05T01:15:34.956 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-05T01:15:35.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:34 smithi183 ceph-mon[34589]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 5.1 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-05T01:15:35.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:34 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/148213721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:35.201 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:34 smithi129 ceph-mon[30979]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 5.1 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-05T01:15:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:34 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/148213721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:34 smithi194 ceph-mon[34941]: pgmap v358: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 8 op/s; 5.1 KiB/s, 0 keys/s, 0 objects/s recovering 2024-09-05T01:15:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:34 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/148213721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:15:36.380 INFO:teuthology.orchestra.run.smithi129.stdout:{"run_dir":"/var/run/ceph"} 2024-09-05T01:15:36.381 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-05T01:15:36.381 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1743 2024-09-05T01:15:36.381 DEBUG:tasks.quiescer:Yielding 2024-09-05T01:15:36.381 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T01:15:36.391 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T01:15:36.391 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:15:36.391 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-05T01:15:36.391 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:36.428 INFO:teuthology.orchestra.run.smithi129.stdout:total 0 2024-09-05T01:15:36.439 INFO:teuthology.orchestra.run.smithi129.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-05T01:15:36.440 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-fuse 1.1T 392M 1.1T 1% /tmp/cephfs 2024-09-05T01:15:36.442 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-05T01:15:36.535 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-05T01:15:36.575 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-05T01:15:36.647 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:36.894 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:36 smithi129 ceph-mon[30979]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 -- 172.21.15.129:0/2293052682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81036e0 msgr2=0x7f48a8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 --2- 172.21.15.129:0/2293052682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81036e0 0x7f48a8107b90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4898009930 tx=0x7f489802a420 comp rx=0 tx=0).stop 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 -- 172.21.15.129:0/2293052682 shutdown_connections 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 --2- 172.21.15.129:0/2293052682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a81082a0 0x7f48a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 --2- 172.21.15.129:0/2293052682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81036e0 0x7f48a8107b90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 --2- 172.21.15.129:0/2293052682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a8102d30 0x7f48a8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 -- 172.21.15.129:0/2293052682 >> 172.21.15.129:0/2293052682 conn(0x7f48a80fc7d0 msgr2=0x7f48a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 -- 172.21.15.129:0/2293052682 shutdown_connections 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.124+0000 7f48af80b640 1 -- 172.21.15.129:0/2293052682 wait complete. 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 Processor -- start 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 -- start start 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 0x7f48a81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a81036e0 0x7f48a8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8111f10 con 0x7f48a81082a0 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8111d90 con 0x7f48a81036e0 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48af80b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f48a8112090 con 0x7f48a8102d30 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:37.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38018/0 (socket says 172.21.15.129:38018) 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48ad5a7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 0x7f48a81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 -- 172.21.15.129:0/648610936 learned_addr learned my addr 172.21.15.129:0/648610936 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48acda6640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a81036e0 0x7f48a8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 -- 172.21.15.129:0/648610936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 msgr2=0x7f48a81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 0x7f48a81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 -- 172.21.15.129:0/648610936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a81036e0 msgr2=0x7f48a8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a81036e0 0x7f48a8198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.125+0000 7f48adda8640 1 -- 172.21.15.129:0/648610936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a819d4a0 con 0x7f48a81082a0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f48ad5a7640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 0x7f48a81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f48adda8640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f48a000cad0 tx=0x7f48a000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a000ad60 con 0x7f48a81082a0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f48a0005020 con 0x7f48a81082a0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a819d790 con 0x7f48a81082a0 2024-09-05T01:15:37.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a000f6b0 con 0x7f48a81082a0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.126+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a81a5070 con 0x7f48a81082a0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.127+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a8103110 con 0x7f48a81082a0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.129+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f48a00051d0 con 0x7f48a81082a0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.130+0000 7f489e7fc640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 0x7f48940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.130+0000 7f48ad5a7640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 0x7f48940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.130+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f48a00cf970 con 0x7f48a81082a0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.130+0000 7f48ad5a7640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 0x7f48940a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48a400afb0 tx=0x7f48a400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:37.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.132+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48a0014030 con 0x7f48a81082a0 2024-09-05T01:15:37.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:36 smithi183 ceph-mon[34589]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:36 smithi194 ceph-mon[34941]: pgmap v359: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:37.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.341+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48a8199850 con 0x7f48a81082a0 2024-09-05T01:15:37.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.342+0000 7f489e7fc640 1 -- 172.21.15.129:0/648610936 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11930 (secure 0 0 0) 0x7f48a8199850 con 0x7f48a81082a0 2024-09-05T01:15:37.342 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:37.343 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":26,"btime":"2024-09-05T01:15:24:939831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:24.411921+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:15:37.343 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 26 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 msgr2=0x7f48940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 0x7f48940a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48a400afb0 tx=0x7f48a400a2d0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 msgr2=0x7f48a819cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f48a000cad0 tx=0x7f48a000ced0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 shutdown_connections 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48940a2c10 0x7f48940a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48a81082a0 0x7f48a819cdc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f48a81036e0 0x7f48a8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 --2- 172.21.15.129:0/648610936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f48a8102d30 0x7f48a81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 >> 172.21.15.129:0/648610936 conn(0x7f48a80fc7d0 msgr2=0x7f48a8105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 shutdown_connections 2024-09-05T01:15:37.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:37.344+0000 7f48af80b640 1 -- 172.21.15.129:0/648610936 wait complete. 2024-09-05T01:15:38.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:37 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/648610936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:37 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/648610936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:38.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:37 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/648610936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:15:38.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.447+0000 7fe4ce05e640 1 -- 172.21.15.129:0/672364065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8101b60 msgr2=0x7fe4c8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.447+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/672364065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8101b60 0x7fe4c8101f60 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009ca0 tx=0x7fe4b802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 -- 172.21.15.129:0/672364065 shutdown_connections 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/672364065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8103710 0x7fe4c8109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/672364065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 0x7fe4c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/672364065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8101b60 0x7fe4c8101f60 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 -- 172.21.15.129:0/672364065 >> 172.21.15.129:0/672364065 conn(0x7fe4c80fd2e0 msgr2=0x7fe4c80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 -- 172.21.15.129:0/672364065 shutdown_connections 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 -- 172.21.15.129:0/672364065 wait complete. 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 Processor -- start 2024-09-05T01:15:38.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.448+0000 7fe4ce05e640 1 -- start start 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8101b60 0x7fe4c8075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 0x7fe4c80745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810d6f0 con 0x7fe4c8103710 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810d570 con 0x7fe4c8102d50 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4ce05e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c810d870 con 0x7fe4c8101b60 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38044/0 (socket says 172.21.15.129:38044) 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 -- 172.21.15.129:0/833680552 learned_addr learned my addr 172.21.15.129:0/833680552 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 0x7fe4c80745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8101b60 0x7fe4c8075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 -- 172.21.15.129:0/833680552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8101b60 msgr2=0x7fe4c8075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8101b60 0x7fe4c8075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 -- 172.21.15.129:0/833680552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 msgr2=0x7fe4c80745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 0x7fe4c80745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 -- 172.21.15.129:0/833680552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c80753b0 con 0x7fe4c8103710 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c7fff640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009fd0 tx=0x7fe4c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.449+0000 7fe4c4ff9640 1 -- 172.21.15.129:0/833680552 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0011830 con 0x7fe4c8103710 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.450+0000 7fe4c4ff9640 1 -- 172.21.15.129:0/833680552 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c0016070 con 0x7fe4c8103710 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.450+0000 7fe4c4ff9640 1 -- 172.21.15.129:0/833680552 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0010de0 con 0x7fe4c8103710 2024-09-05T01:15:38.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.450+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c81a1ea0 con 0x7fe4c8103710 2024-09-05T01:15:38.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.450+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a22b0 con 0x7fe4c8103710 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.452+0000 7fe4c4ff9640 1 -- 172.21.15.129:0/833680552 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fe4c00119d0 con 0x7fe4c8103710 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.452+0000 7fe4d0dfa740 1 Processor -- start 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.452+0000 7fe4d0dfa740 1 -- start start 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f22e0 0x5635791f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635791f9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635791552f0 con 0x5635791f7370 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635790fd270 con 0x5635791f19a0 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5635791512b0 con 0x5635791f22e0 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635791f9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:60956/0 (socket says 172.21.15.129:60956) 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/2111242330 learned_addr learned my addr 172.21.15.129:0/2111242330 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f22e0 0x5635791f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c4ff9640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 0x7fe4b00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/2111242330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f22e0 msgr2=0x5635791f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f22e0 0x5635791f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/2111242330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 msgr2=0x5635791f9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635791f9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/2111242330 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635790fc340 con 0x5635791f19a0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c7fff640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 0x7fe4b00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c4ff9640 1 -- 172.21.15.129:0/833680552 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe4c00ceb10 con 0x7fe4c8103710 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc012790 tx=0x7fe4bc00b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16e0c937a418e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.453+0000 7fe4adffb640 1 -- 172.21.15.129:0/2111242330 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004050 con 0x5635791f19a0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4c7fff640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 0x7fe4b00a5190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe4c009aea0 tx=0x7fe4c0069e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4adffb640 1 -- 172.21.15.129:0/2111242330 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4bc0054d0 con 0x5635791f19a0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4adffb640 1 -- 172.21.15.129:0/2111242330 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004b20 con 0x5635791f19a0 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/2111242330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 msgr2=0x5635791f1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc012790 tx=0x7fe4bc00b390 comp rx=0 tx=0).stop 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/2111242330 shutdown_connections 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635791f9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f22e0 0x5635791f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/2111242330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f19a0 0x5635791f1da0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/2111242330 >> 172.21.15.129:0/2111242330 conn(0x5635791ed010 msgr2=0x5635791ed410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/2111242330 shutdown_connections 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.454+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/2111242330 wait complete. 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 -- start start 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 Processor -- start 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635791fa340 con 0x5635791f7370 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635790de290 con 0x5635791f22e0 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5635791fa4c0 con 0x5635791f19a0 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4d0dfa740 10 client.0 _ll_register_callbacks cb 0x56357910f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:60964/0 (socket says 172.21.15.129:60964) 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38058/0 (socket says 172.21.15.129:38058) 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c77fe640 1 -- 172.21.15.129:0/3655458285 learned_addr learned my addr 172.21.15.129:0/3655458285 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.455+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/3655458285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 msgr2=0x5635791fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 msgr2=0x5635791fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 -- 172.21.15.129:0/3655458285 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563578e68210 con 0x5635791f7370 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0057c0 tx=0x7fe4bc017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744b98c14d06ebf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc023070 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe4bc0052d0 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4bc0054d0 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4bc035050 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 10 client.15312 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563578f34620 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4bc0054d0 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4bc00fa30 con 0x5635791f7370 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 10 client.15312 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 10 client.15312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 20 client.15312 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635792075d0 0x563579209a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 4 client.15312 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.456+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563579162a20 con 0x5635792075d0 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.457+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635792075d0 0x563579209a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.457+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635792075d0 0x563579209a10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a3e304b08b20be7 server_cookie=b32fe795cf5d785a in_seq=0 out_seq=0 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.457+0000 7fe4a6ffd640 10 client.15312 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/3655458285 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563579162a20 con 0x5635792075d0 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4a6ffd640 10 client.15312 handle_command_reply: tid=0 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 client.15312 shutdown 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 msgr2=0x5635792030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0057c0 tx=0x7fe4bc017040 comp rx=0 tx=0).stop 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 shutdown_connections 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635792075d0 0x563579209a10 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f7370 0x5635792030f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f22e0 0x5635791fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4a6ffd640 0 client.15312 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/3655458285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 >> 172.21.15.129:0/3655458285 conn(0x5635791ed010 msgr2=0x5635791ee820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 shutdown_connections 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/3655458285 wait complete. 2024-09-05T01:15:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.458+0000 7fe4d0dfa740 20 client.15312 trim_cache size 0 max 16384 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 Processor -- start 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 -- start start 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81af340 con 0x5635791f22e0 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81acc10 con 0x5635791f7370 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81aaa60 con 0x5635791f19a0 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38060/0 (socket says 172.21.15.129:38060) 2024-09-05T01:15:38.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 -- 172.21.15.129:0/398373918 learned_addr learned my addr 172.21.15.129:0/398373918 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 -- 172.21.15.129:0/398373918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 msgr2=0x5635791fbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 -- 172.21.15.129:0/398373918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 msgr2=0x5635791fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c77fe640 1 -- 172.21.15.129:0/398373918 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635791f27c0 con 0x5635791f22e0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.472+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009ca0 tx=0x7fe4b8031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda0a604859d780 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4ad7fa640 1 -- 172.21.15.129:0/398373918 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8003970 con 0x5635791f22e0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4ad7fa640 1 -- 172.21.15.129:0/398373918 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4b8036040 con 0x5635791f22e0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4ad7fa640 1 -- 172.21.15.129:0/398373918 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8030820 con 0x5635791f22e0 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/398373918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 msgr2=0x563579203080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009ca0 tx=0x7fe4b8031000 comp rx=0 tx=0).stop 2024-09-05T01:15:38.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/398373918 shutdown_connections 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x563579203080 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/398373918 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x5635791fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/398373918 >> 172.21.15.129:0/398373918 conn(0x5635791ed010 msgr2=0x5635791f3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/398373918 shutdown_connections 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.473+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/398373918 wait complete. 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 -- start start 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 Processor -- start 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x563579202ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ab0d0 con 0x5635791f22e0 2024-09-05T01:15:38.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ac540 con 0x5635791f7370 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ab7a0 con 0x5635791f19a0 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4d0dfa740 10 client.0 _ll_register_callbacks cb 0x7fe4bc00be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38064/0 (socket says 172.21.15.129:38064) 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 -- 172.21.15.129:0/1855511448 learned_addr learned my addr 172.21.15.129:0/1855511448 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 -- 172.21.15.129:0/1855511448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 msgr2=0x563579202ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x563579202ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 -- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 msgr2=0x5635791fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.474+0000 7fe4c77fe640 1 -- 172.21.15.129:0/1855511448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635792702a0 con 0x5635791f22e0 2024-09-05T01:15:38.480 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4c77fe640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8000f80 tx=0x7fe4b8004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2766ebc50e98e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b805d020 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe4b80307c0 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b8036040 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4b8055040 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 10 client.15318 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563579271590 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4b8036040 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe4b8058030 con 0x5635791f22e0 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 10 client.15318 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 10 client.15318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 20 client.15318 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563579271fc0 0x5635792743e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:38.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 4 client.15318 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:15:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.475+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56357915cd70 con 0x563579271fc0 2024-09-05T01:15:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.476+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563579271fc0 0x5635792743e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.476+0000 7fe4c6ffd640 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563579271fc0 0x5635792743e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab3c1d1d426f758 server_cookie=a9b1d7ae8ec95f32 in_seq=0 out_seq=0 2024-09-05T01:15:38.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.476+0000 7fe4a6ffd640 10 client.15318 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:38.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.493+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1855511448 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56357915cd70 con 0x563579271fc0 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.493+0000 7fe4a6ffd640 10 client.15318 handle_command_reply: tid=0 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.493+0000 7fe4d0dfa740 1 client.15318 shutdown 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 msgr2=0x5635791ffa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8000f80 tx=0x7fe4b8004890 comp rx=0 tx=0).stop 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 shutdown_connections 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563579271fc0 0x5635792743e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635791f7370 0x5635791fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4a6ffd640 0 client.15318 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635791f22e0 0x5635791ffa90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 --2- 172.21.15.129:0/1855511448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635791f19a0 0x563579202ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 >> 172.21.15.129:0/1855511448 conn(0x5635791ed010 msgr2=0x5635791fbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 shutdown_connections 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 1 -- 172.21.15.129:0/1855511448 wait complete. 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4d0dfa740 20 client.15318 trim_cache size 0 max 16384 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:15:38.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 msgr2=0x7fe4b00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 0x7fe4b00a5190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe4c009aea0 tx=0x7fe4c0069e90 comp rx=0 tx=0).stop 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 msgr2=0x7fe4c8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009fd0 tx=0x7fe4c000b8e0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 shutdown_connections 2024-09-05T01:15:38.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4b00a2cd0 0x7fe4b00a5190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4c8103710 0x7fe4c8074b20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe4c8102d50 0x7fe4c80745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 --2- 172.21.15.129:0/833680552 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe4c8101b60 0x7fe4c8075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.494+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 >> 172.21.15.129:0/833680552 conn(0x7fe4c80fd2e0 msgr2=0x7fe4c8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.495+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 shutdown_connections 2024-09-05T01:15:38.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:38.495+0000 7fe4ce05e640 1 -- 172.21.15.129:0/833680552 wait complete. 2024-09-05T01:15:38.692 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-05T01:15:38.770 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-05T01:15:38.772 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-05T01:15:38.850 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-05T01:15:38.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T01:13:06.500509+0000', 'modified': '2024-09-05T01:15:24.411921+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24452, 'mds_2': 24451}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/3903364483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 3903364483}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 3903364483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6835/1641200590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6834', 'nonce': 1641200590}, {'type': 'v1', 'addr': '172.21.15.194:6835', 'nonce': 1641200590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.194:6837/979076613', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6836', 'nonce': 979076613}, {'type': 'v1', 'addr': '172.21.15.194:6837', 'nonce': 979076613}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.194:6839/203556383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.194:6838', 'nonce': 203556383}, {'type': 'v1', 'addr': '172.21.15.194:6839', 'nonce': 203556383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.183:6837/685319691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 685319691}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 685319691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/3533858614', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 3533858614}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 3533858614}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24488, 'qdb_cluster': [24488, 24452, 24451]} 2024-09-05T01:15:38.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-05T01:15:38.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T01:15:38.955 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/: 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:total 2 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 .. 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 data 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data: 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:38.956 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 .. 2024-09-05T01:15:38.958 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-05T01:15:39.020 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:38 smithi129 ceph-mon[30979]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:39.042 INFO:teuthology.orchestra.run.smithi129.stdout:[Service] 2024-09-05T01:15:39.042 INFO:teuthology.orchestra.run.smithi129.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-05T01:15:39.043 INFO:teuthology.orchestra.run.smithi129.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-05T01:15:39.045 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-05T01:15:39.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:38 smithi183 ceph-mon[34589]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:39.174 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:39.183 INFO:teuthology.orchestra.run.smithi129.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-05T01:15:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:38 smithi194 ceph-mon[34941]: pgmap v360: 129 pgs: 129 active+clean; 650 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 -- 172.21.15.129:0/3920942692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538100e20 msgr2=0x7f1538101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 --2- 172.21.15.129:0/3920942692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538100e20 0x7f1538101200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1528009900 tx=0x7f152802a480 comp rx=0 tx=0).stop 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 -- 172.21.15.129:0/3920942692 shutdown_connections 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 --2- 172.21.15.129:0/3920942692 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538102ef0 0x7f153810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 --2- 172.21.15.129:0/3920942692 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f15381029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 --2- 172.21.15.129:0/3920942692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538100e20 0x7f1538101200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 -- 172.21.15.129:0/3920942692 >> 172.21.15.129:0/3920942692 conn(0x7f15380fc7d0 msgr2=0x7f15380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 -- 172.21.15.129:0/3920942692 shutdown_connections 2024-09-05T01:15:39.736 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.734+0000 7f153db5b640 1 -- 172.21.15.129:0/3920942692 wait complete. 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 Processor -- start 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 -- start start 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100e20 0x7f1538103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f1538103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1538110e30 con 0x7f1538102ef0 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1538110cb0 con 0x7f1538100e20 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f153db5b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1538110fb0 con 0x7f1538102570 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38088/0 (socket says 172.21.15.129:38088) 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100e20 0x7f1538103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f1537fff640 1 -- 172.21.15.129:0/3833821785 learned_addr learned my addr 172.21.15.129:0/3833821785 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.735+0000 7f152ffff640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f1538103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 -- 172.21.15.129:0/3833821785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 msgr2=0x7f1538103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f1538103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 -- 172.21.15.129:0/3833821785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100e20 msgr2=0x7f1538103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100e20 0x7f1538103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.739 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 -- 172.21.15.129:0/3833821785 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15381a6990 con 0x7f1538102ef0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f152ffff640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f1538103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f1537fff640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1530009fd0 tx=0x7f153000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f15357fa640 1 -- 172.21.15.129:0/3833821785 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530017070 con 0x7f1538102ef0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f15357fa640 1 -- 172.21.15.129:0/3833821785 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530003e20 con 0x7f1538102ef0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15381a6c80 con 0x7f1538102ef0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381a7170 con 0x7f1538102ef0 2024-09-05T01:15:39.740 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.736+0000 7f15357fa640 1 -- 172.21.15.129:0/3833821785 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530010460 con 0x7f1538102ef0 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.739+0000 7f15357fa640 1 -- 172.21.15.129:0/3833821785 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1530003970 con 0x7f1538102ef0 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.739+0000 7f15357fa640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 0x7f15240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.739+0000 7f15357fa640 1 -- 172.21.15.129:0/3833821785 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f15300cf3a0 con 0x7f1538102ef0 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.739+0000 7f15377fe640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 0x7f15240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 Processor -- start 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 -- start start 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d62db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f15377fe640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 0x7f15240a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1528009230 tx=0x7f15280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.743 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4cc9090 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4c70a70 con 0x55d0e4d632f0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f153ff34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4cc5050 con 0x55d0e4d629d0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38098/0 (socket says 172.21.15.129:38098) 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 -- 172.21.15.129:0/2172554571 learned_addr learned my addr 172.21.15.129:0/2172554571 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f152ffff640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f15377fe640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d62db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 -- 172.21.15.129:0/2172554571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 msgr2=0x55d0e4d62db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.744 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d62db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.740+0000 7f1537fff640 1 -- 172.21.15.129:0/2172554571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 msgr2=0x55d0e4d6d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f1537fff640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f1537fff640 1 -- 172.21.15.129:0/2172554571 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e4c7d130 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f152ffff640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f1537fff640 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f153009b620 tx=0x7f153009b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91143a7149c359ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f152dffb640 1 -- 172.21.15.129:0/2172554571 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530020070 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f152dffb640 1 -- 172.21.15.129:0/2172554571 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153009bd90 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.741+0000 7f152dffb640 1 -- 172.21.15.129:0/2172554571 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530099a60 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- 172.21.15.129:0/2172554571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 msgr2=0x55d0e4d6ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f153009b620 tx=0x7f153009b580 comp rx=0 tx=0).stop 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- 172.21.15.129:0/2172554571 shutdown_connections 2024-09-05T01:15:39.745 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6ffa0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 --2- 172.21.15.129:0/2172554571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d62db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- 172.21.15.129:0/2172554571 >> 172.21.15.129:0/2172554571 conn(0x55d0e4d5ef60 msgr2=0x55d0e4d5f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- 172.21.15.129:0/2172554571 shutdown_connections 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- 172.21.15.129:0/2172554571 wait complete. 2024-09-05T01:15:39.746 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.742+0000 7f153ff34740 1 -- start start 2024-09-05T01:15:39.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 Processor -- start 2024-09-05T01:15:39.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d6d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.749 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d70780 con 0x55d0e4d632f0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4c83f20 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d70900 con 0x55d0e4d629d0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f153ff34740 10 client.0 _ll_register_callbacks cb 0x55d0e4c4d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38104/0 (socket says 172.21.15.129:38104) 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 -- 172.21.15.129:0/295446322 learned_addr learned my addr 172.21.15.129:0/295446322 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f15377fe640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d6d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 -- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 msgr2=0x55d0e4d6d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d6d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 -- 172.21.15.129:0/295446322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 msgr2=0x55d0e4d6adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f1537fff640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f152ffff640 1 -- 172.21.15.129:0/295446322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e4a3a2b0 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.743+0000 7f1537fff640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.744+0000 7f152ffff640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1520010840 tx=0x7f1520010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ac5765c08d8d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.744+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.744+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152000ed40 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.744+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1520005020 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.744+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f153009bd90 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f152002c050 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0e4d7c1b0 con 0x55d0e4d632f0 2024-09-05T01:15:39.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f153009bd90 con 0x55d0e4d632f0 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1520021330 con 0x55d0e4d632f0 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 10 client.15330 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 20 client.15330 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4d7e780 0x55d0e4d7fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 4 client.15330 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0e4cd6600 con 0x55d0e4d7e780 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.745+0000 7f15377fe640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4d7e780 0x55d0e4d7fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.746+0000 7f15377fe640 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4d7e780 0x55d0e4d7fbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98a0df9df0f0e5c6 server_cookie=c2e457072e8d156c in_seq=0 out_seq=0 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.746+0000 7f1516ffd640 10 client.15330 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f1516ffd640 1 -- 172.21.15.129:0/295446322 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0e4cd6600 con 0x55d0e4d7e780 2024-09-05T01:15:39.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f1516ffd640 10 client.15330 handle_command_reply: tid=0 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 client.15330 shutdown 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 msgr2=0x55d0e4d6a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f1520010840 tx=0x7f1520010d00 comp rx=0 tx=0).stop 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 shutdown_connections 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4d7e780 0x55d0e4d7fbf0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a850 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 --2- 172.21.15.129:0/295446322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d629d0 0x55d0e4d6d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f1516ffd640 0 client.15330 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 >> 172.21.15.129:0/295446322 conn(0x55d0e4d5ef60 msgr2=0x55d0e4d64c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 shutdown_connections 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 1 -- 172.21.15.129:0/295446322 wait complete. 2024-09-05T01:15:39.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.747+0000 7f153ff34740 20 client.15330 trim_cache size 0 max 16384 2024-09-05T01:15:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.768+0000 7f153ff34740 1 Processor -- start 2024-09-05T01:15:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.768+0000 7f153ff34740 1 -- start start 2024-09-05T01:15:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.768+0000 7f153ff34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f153ff34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f153ff34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f153ff34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d7df30 con 0x55d0e4d629d0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f153ff34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d7e600 con 0x55d0e4d632f0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f153ff34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d7ecd0 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38110/0 (socket says 172.21.15.129:38110) 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 -- 172.21.15.129:0/1177080724 learned_addr learned my addr 172.21.15.129:0/1177080724 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f152ffff640 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f1537fff640 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 -- 172.21.15.129:0/1177080724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d6dbb0 msgr2=0x55d0e4d6a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 -- 172.21.15.129:0/1177080724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 msgr2=0x55d0e4d6a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 -- 172.21.15.129:0/1177080724 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0e4d70890 con 0x55d0e4d629d0 2024-09-05T01:15:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f15377fe640 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f152802a9a0 tx=0x7f1528040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8f41b2260a74be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f152d7fa640 1 -- 172.21.15.129:0/1177080724 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528035040 con 0x55d0e4d629d0 2024-09-05T01:15:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.769+0000 7f152d7fa640 1 -- 172.21.15.129:0/1177080724 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1528057070 con 0x55d0e4d629d0 2024-09-05T01:15:39.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f152d7fa640 1 -- 172.21.15.129:0/1177080724 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1528051570 con 0x55d0e4d629d0 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 -- 172.21.15.129:0/1177080724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 msgr2=0x55d0e4d7ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f152802a9a0 tx=0x7f1528040000 comp rx=0 tx=0).stop 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 -- 172.21.15.129:0/1177080724 shutdown_connections 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d6a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d632f0 0x55d0e4d6a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 --2- 172.21.15.129:0/1177080724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d629d0 0x55d0e4d7ee50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.770+0000 7f153ff34740 1 -- 172.21.15.129:0/1177080724 >> 172.21.15.129:0/1177080724 conn(0x55d0e4d5ef60 msgr2=0x55d0e4d69140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- 172.21.15.129:0/1177080724 shutdown_connections 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- 172.21.15.129:0/1177080724 wait complete. 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- start start 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 Processor -- start 2024-09-05T01:15:39.776 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d629d0 0x55d0e4d887b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 0x55d0e4d67890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d7fa70 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d7f3a0 con 0x55d0e4d629d0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d0e4d80140 con 0x55d0e4d632f0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.771+0000 7f153ff34740 10 client.0 _ll_register_callbacks cb 0x7f152000c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38126/0 (socket says 172.21.15.129:38126) 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 -- 172.21.15.129:0/373557035 learned_addr learned my addr 172.21.15.129:0/373557035 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f15377fe640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d629d0 0x55d0e4d887b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f152ffff640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 0x55d0e4d67890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 -- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 msgr2=0x55d0e4d67890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 0x55d0e4d67890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 -- 172.21.15.129:0/373557035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d629d0 msgr2=0x55d0e4d887b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d629d0 0x55d0e4d887b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 -- 172.21.15.129:0/373557035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0e4d86290 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f152ffff640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 0x55d0e4d67890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1537fff640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1530005c40 tx=0x7f1530004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23464de02c2cca3b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15300b9060 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f15300cf170 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.772+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1528057070 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15300b3030 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.778 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 10 client.15336 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d0e4de9070 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1528057070 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1530097d00 con 0x55d0e4d6dbb0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 10 client.15336 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 10 client.15336 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 20 client.15336 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4ded6d0 0x55d0e4dec3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 4 client.15336 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 --> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] -- command(tid 0: {"prefix": "status"}) -- 0x55d0e4cd0b10 con 0x55d0e4ded6d0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.773+0000 7f15377fe640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4ded6d0 0x55d0e4dec3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f15377fe640 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4ded6d0 0x55d0e4dec3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80bd69f923fd5c93 server_cookie=6ae0c4146950e14d in_seq=0 out_seq=0 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f1516ffd640 10 client.15336 ms_handle_connect on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:39.779 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f1516ffd640 1 -- 172.21.15.129:0/373557035 <== mds.2 v2:172.21.15.194:6838/203556383 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0e4cd0b10 con 0x55d0e4ded6d0 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f1516ffd640 10 client.15336 handle_command_reply: tid=0 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 client.15336 shutdown 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 msgr2=0x55d0e4d68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1530005c40 tx=0x7f1530004280 comp rx=0 tx=0).stop 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 shutdown_connections 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:6838/203556383,v1:172.21.15.194:6839/203556383] conn(0x55d0e4ded6d0 0x55d0e4dec3e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d0e4d6dbb0 0x55d0e4d68020 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d0e4d632f0 0x55d0e4d67890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f1516ffd640 0 client.15336 ms_handle_reset on v2:172.21.15.194:6838/203556383 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 --2- 172.21.15.129:0/373557035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d0e4d629d0 0x55d0e4d887b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 >> 172.21.15.129:0/373557035 conn(0x55d0e4d5ef60 msgr2=0x55d0e4d65500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 shutdown_connections 2024-09-05T01:15:39.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.774+0000 7f153ff34740 1 -- 172.21.15.129:0/373557035 wait complete. 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153ff34740 20 client.15336 trim_cache size 0 max 16384 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 2, 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24451, 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:39.781 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 111.929038259, 2024-09-05T01:15:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 26, 2024-09-05T01:15:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 90, 2024-09-05T01:15:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T01:15:39.782 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 172.13406671600001 2024-09-05T01:15:39.782 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 msgr2=0x7f15240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 0x7f15240a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1528009230 tx=0x7f15280058d0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 msgr2=0x7f15381041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1530009fd0 tx=0x7f153000b260 comp rx=0 tx=0).stop 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 shutdown_connections 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15240a2c10 0x7f15240a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538102ef0 0x7f15381041d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538102570 0x7f1538103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 --2- 172.21.15.129:0/3833821785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100e20 0x7f1538103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:39.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 >> 172.21.15.129:0/3833821785 conn(0x7f15380fc7d0 msgr2=0x7f153806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:39.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.775+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 shutdown_connections 2024-09-05T01:15:39.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:39.776+0000 7f153db5b640 1 -- 172.21.15.129:0/3833821785 wait complete. 2024-09-05T01:15:40.481 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T01:15:40.757 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:41.154 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:40 smithi129 ceph-mon[30979]: pgmap v361: 129 pgs: 129 active+clean; 657 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-05T01:15:41.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:40 smithi183 ceph-mon[34589]: pgmap v361: 129 pgs: 129 active+clean; 657 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-05T01:15:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:40 smithi194 ceph-mon[34941]: pgmap v361: 129 pgs: 129 active+clean; 657 KiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-09-05T01:15:41.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 -- 172.21.15.129:0/2637900312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 msgr2=0x7f1514103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 --2- 172.21.15.129:0/2637900312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514103170 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f1510009930 tx=0x7f151002a420 comp rx=0 tx=0).stop 2024-09-05T01:15:41.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 -- 172.21.15.129:0/2637900312 shutdown_connections 2024-09-05T01:15:41.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 --2- 172.21.15.129:0/2637900312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.255 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 --2- 172.21.15.129:0/2637900312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 0x7f1514107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 --2- 172.21.15.129:0/2637900312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514103170 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 -- 172.21.15.129:0/2637900312 >> 172.21.15.129:0/2637900312 conn(0x7f15140fc7d0 msgr2=0x7f15140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 -- 172.21.15.129:0/2637900312 shutdown_connections 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.253+0000 7f151be97640 1 -- 172.21.15.129:0/2637900312 wait complete. 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 Processor -- start 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 -- start start 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 0x7f1514198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111f40 con 0x7f1514102d90 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1514111dc0 con 0x7f1514103740 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151be97640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f15141120c0 con 0x7f1514108290 2024-09-05T01:15:41.256 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38154/0 (socket says 172.21.15.129:38154) 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f1519c33640 1 -- 172.21.15.129:0/1408594721 learned_addr learned my addr 172.21.15.129:0/1408594721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.254+0000 7f1519432640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 0x7f1514198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 -- 172.21.15.129:0/1408594721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 msgr2=0x7f1514198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 0x7f1514198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 -- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 msgr2=0x7f1514198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.257 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 -- 172.21.15.129:0/1408594721 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151419d4b0 con 0x7f1514108290 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.255+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151419cdd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f150c009fd0 tx=0x7f150c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.256+0000 7f150affd640 1 -- 172.21.15.129:0/1408594721 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c00ce20 con 0x7f1514108290 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.256+0000 7f150affd640 1 -- 172.21.15.129:0/1408594721 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f150c005020 con 0x7f1514108290 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.256+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151419d7a0 con 0x7f1514108290 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.256+0000 7f150affd640 1 -- 172.21.15.129:0/1408594721 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c010690 con 0x7f1514108290 2024-09-05T01:15:41.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.256+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1514100920 con 0x7f1514108290 2024-09-05T01:15:41.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.259+0000 7f150affd640 1 -- 172.21.15.129:0/1408594721 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f150c0051d0 con 0x7f1514108290 2024-09-05T01:15:41.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.259+0000 7f151e270740 1 Processor -- start 2024-09-05T01:15:41.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.259+0000 7f151e270740 1 -- start start 2024-09-05T01:15:41.260 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 0x562b4a18a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a18abb0 0x562b4a18cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b4a0e6090 con 0x562b4a17f9d0 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4a08da70 con 0x562b4a1802f0 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151e270740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b4a0e2050 con 0x562b4a18abb0 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f150affd640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 0x7f15000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38160/0 (socket says 172.21.15.129:38160) 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 -- 172.21.15.129:0/1015482293 learned_addr learned my addr 172.21.15.129:0/1015482293 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 0x7f15000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519c33640 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 0x562b4a18a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 -- 172.21.15.129:0/1015482293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a18abb0 msgr2=0x562b4a18cfa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a18abb0 0x562b4a18cfa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 -- 172.21.15.129:0/1015482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 msgr2=0x562b4a18a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 0x562b4a18a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519432640 1 -- 172.21.15.129:0/1015482293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4a09a130 con 0x562b4a17f9d0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.260+0000 7f1519c33640 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 0x562b4a18a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f150affd640 1 -- 172.21.15.129:0/1408594721 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f150c0cfcb0 con 0x7f1514108290 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f151a434640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 0x7f15000a5150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f150c09bed0 tx=0x7f150c097be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f1519432640 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1504010cb0 tx=0x7f1504009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2b9258da7f9e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f14fffff640 1 -- 172.21.15.129:0/1015482293 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504004220 con 0x562b4a17f9d0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f14fffff640 1 -- 172.21.15.129:0/1015482293 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15040043c0 con 0x562b4a17f9d0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.261+0000 7f14fffff640 1 -- 172.21.15.129:0/1015482293 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150401b8e0 con 0x562b4a17f9d0 2024-09-05T01:15:41.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- 172.21.15.129:0/1015482293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 msgr2=0x562b4a17fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1504010cb0 tx=0x7f1504009900 comp rx=0 tx=0).stop 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- 172.21.15.129:0/1015482293 shutdown_connections 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a18abb0 0x562b4a18cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1802f0 0x562b4a18a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 --2- 172.21.15.129:0/1015482293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a17f9d0 0x562b4a17fdb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- 172.21.15.129:0/1015482293 >> 172.21.15.129:0/1015482293 conn(0x562b4a17bf60 msgr2=0x562b4a17c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- 172.21.15.129:0/1015482293 shutdown_connections 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- 172.21.15.129:0/1015482293 wait complete. 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 -- start start 2024-09-05T01:15:41.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.262+0000 7f151e270740 1 Processor -- start 2024-09-05T01:15:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a187970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1883f0 0x562b4a196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b4a18d780 con 0x562b4a18abb0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4a0a0f20 con 0x562b4a1883f0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b4a18d900 con 0x562b4a1802f0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f151e270740 10 client.0 _ll_register_callbacks cb 0x562b4a06a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38176/0 (socket says 172.21.15.129:38176) 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f1519c33640 1 -- 172.21.15.129:0/1448274769 learned_addr learned my addr 172.21.15.129:0/1448274769 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f1519432640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a187970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.263+0000 7f1519432640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1883f0 0x562b4a196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 -- 172.21.15.129:0/1448274769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 msgr2=0x562b4a187970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a187970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 -- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1883f0 msgr2=0x562b4a196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1883f0 0x562b4a196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 -- 172.21.15.129:0/1448274769 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b49e572b0 con 0x562b4a18abb0 2024-09-05T01:15:41.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f1519c33640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1510009230 tx=0x7f1510009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35a344980d77ee8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f14fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510035040 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15040043c0 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f15100316a0 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1510055050 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f151e270740 10 client.15348 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b4a1996d0 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.264+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15040043c0 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1510003b10 con 0x562b4a18abb0 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 10 client.15348 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 10 client.15348 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 20 client.15348 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a19bcd0 0x562b4a19d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 4 client.15348 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4a0f3600 con 0x562b4a19bcd0 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.265+0000 7f1519432640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a19bcd0 0x562b4a19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f1519432640 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a19bcd0 0x562b4a19d140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e21606d71fe1b931 server_cookie=f507552394b3943c in_seq=0 out_seq=0 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f14fd7fa640 10 client.15348 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f14fd7fa640 1 -- 172.21.15.129:0/1448274769 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b4a0f3600 con 0x562b4a19bcd0 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f14fd7fa640 10 client.15348 handle_command_reply: tid=0 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f151e270740 1 client.15348 shutdown 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 msgr2=0x562b4a187eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.266+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1510009230 tx=0x7f1510009630 comp rx=0 tx=0).stop 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 shutdown_connections 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a19bcd0 0x562b4a19d140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a1883f0 0x562b4a196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a18abb0 0x562b4a187eb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f14fd7fa640 0 client.15348 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 --2- 172.21.15.129:0/1448274769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a187970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 >> 172.21.15.129:0/1448274769 conn(0x562b4a17bf60 msgr2=0x562b4a181c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 shutdown_connections 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 1 -- 172.21.15.129:0/1448274769 wait complete. 2024-09-05T01:15:41.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.267+0000 7f151e270740 20 client.15348 trim_cache size 0 max 16384 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 Processor -- start 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 -- start start 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 0x562b4a196b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 0x562b4a181470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a1819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19b480 con 0x562b4a1802f0 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19bb50 con 0x562b4a18abb0 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f151e270740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19c220 con 0x562b4a1883f0 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 0x562b4a196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519432640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 0x562b4a196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38186/0 (socket says 172.21.15.129:38186) 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519432640 1 -- 172.21.15.129:0/360659291 learned_addr learned my addr 172.21.15.129:0/360659291 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519c33640 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 0x562b4a181470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519432640 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a1819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519c33640 1 -- 172.21.15.129:0/360659291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 msgr2=0x562b4a1819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.291+0000 7f1519c33640 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a1819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f1519c33640 1 -- 172.21.15.129:0/360659291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 msgr2=0x562b4a196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f1519c33640 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 0x562b4a196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f1519c33640 1 -- 172.21.15.129:0/360659291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4a18d890 con 0x562b4a1883f0 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f1519c33640 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 0x562b4a181470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1510031440 tx=0x7f151002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1931c54fc07df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f14ff7fe640 1 -- 172.21.15.129:0/360659291 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151005e070 con 0x562b4a1883f0 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f14ff7fe640 1 -- 172.21.15.129:0/360659291 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1510009780 con 0x562b4a1883f0 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.292+0000 7f14ff7fe640 1 -- 172.21.15.129:0/360659291 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510035070 con 0x562b4a1883f0 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- 172.21.15.129:0/360659291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 msgr2=0x562b4a181470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 0x562b4a181470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1510031440 tx=0x7f151002a420 comp rx=0 tx=0).stop 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- 172.21.15.129:0/360659291 shutdown_connections 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a1819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1883f0 0x562b4a181470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 --2- 172.21.15.129:0/360659291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1802f0 0x562b4a196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- 172.21.15.129:0/360659291 >> 172.21.15.129:0/360659291 conn(0x562b4a17bf60 msgr2=0x562b4a186240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- 172.21.15.129:0/360659291 shutdown_connections 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- 172.21.15.129:0/360659291 wait complete. 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 -- start start 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.293+0000 7f151e270740 1 Processor -- start 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a1a5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19cfc0 con 0x562b4a1883f0 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19c8f0 con 0x562b4a18abb0 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562b4a19d690 con 0x562b4a1802f0 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f151e270740 10 client.0 _ll_register_callbacks cb 0x7f150c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f1519432640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a1a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f1519c33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38190/0 (socket says 172.21.15.129:38190) 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f1519c33640 1 -- 172.21.15.129:0/3150420165 learned_addr learned my addr 172.21.15.129:0/3150420165 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.294+0000 7f1519432640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 -- 172.21.15.129:0/3150420165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 msgr2=0x562b4a1a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a1a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 -- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 msgr2=0x562b4a19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 -- 172.21.15.129:0/3150420165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4a19f4c0 con 0x562b4a1883f0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519c33640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1510005c40 tx=0x7f151003bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee81e9b36c4d9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f14fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510035050 con 0x562b4a1883f0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f15100316a0 con 0x562b4a1883f0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1510009780 con 0x562b4a1883f0 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f1519432640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:15:41.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f151006f050 con 0x562b4a1883f0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f151e270740 10 client.15354 fetch_fsmap learned FSMap version 26 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.295+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b4a1a3f20 con 0x562b4a1883f0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1510009780 con 0x562b4a1883f0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f151006a030 con 0x562b4a1883f0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 10 client.15354 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 20 client.15354 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a20a920 0x562b4a209630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 4 client.15354 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 --> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] -- command(tid 0: {"prefix": "status"}) -- 0x562b4a0edb10 con 0x562b4a20a920 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.296+0000 7f1519432640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a20a920 0x562b4a209630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f14fd7fa640 10 client.15354 ms_handle_connect on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f1519432640 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a20a920 0x562b4a209630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=734e56d96b1825e4 server_cookie=bac0944b87904d13 in_seq=0 out_seq=0 2024-09-05T01:15:41.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f14fd7fa640 1 -- 172.21.15.129:0/3150420165 <== mds.1 v2:172.21.15.183:6836/685319691 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562b4a0edb10 con 0x562b4a20a920 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f14fd7fa640 10 client.15354 handle_command_reply: tid=0 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 client.15354 shutdown 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 msgr2=0x562b4a1a5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1510005c40 tx=0x7f151003bdb0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 shutdown_connections 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:6836/685319691,v1:172.21.15.183:6837/685319691] conn(0x562b4a20a920 0x562b4a209630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562b4a18abb0 0x562b4a19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562b4a1883f0 0x562b4a1a5ca0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f14fd7fa640 0 client.15354 ms_handle_reset on v2:172.21.15.183:6836/685319691 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 --2- 172.21.15.129:0/3150420165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562b4a1802f0 0x562b4a1a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 >> 172.21.15.129:0/3150420165 conn(0x562b4a17bf60 msgr2=0x562b4a17a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 shutdown_connections 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 1 -- 172.21.15.129:0/3150420165 wait complete. 2024-09-05T01:15:41.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.297+0000 7f151e270740 20 client.15354 trim_cache size 0 max 16384 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 1, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24452, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 114.46266414199999, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 26, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 90, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 176.28449533400001 2024-09-05T01:15:41.302 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 msgr2=0x7f15000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 0x7f15000a5150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f150c09bed0 tx=0x7f150c097be0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 msgr2=0x7f151419cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151419cdd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f150c009fd0 tx=0x7f150c00ef10 comp rx=0 tx=0).stop 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 shutdown_connections 2024-09-05T01:15:41.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15000a2c90 0x7f15000a5150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1514108290 0x7f151419cdd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1514103740 0x7f1514198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 --2- 172.21.15.129:0/1408594721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1514102d90 0x7f1514198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 >> 172.21.15.129:0/1408594721 conn(0x7f15140fc7d0 msgr2=0x7f1514105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 shutdown_connections 2024-09-05T01:15:41.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:41.298+0000 7f151be97640 1 -- 172.21.15.129:0/1408594721 wait complete. 2024-09-05T01:15:42.013 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T01:15:42.301 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1964809927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54101dd0 msgr2=0x7f3a541021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1964809927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54101dd0 0x7f3a541021b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3a50009900 tx=0x7f3a5002a410 comp rx=0 tx=0).stop 2024-09-05T01:15:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1964809927 shutdown_connections 2024-09-05T01:15:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1964809927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54106f00 0x7f3a5410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1964809927 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 0x7f3a541067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1964809927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54101dd0 0x7f3a541021b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1964809927 >> 172.21.15.129:0/1964809927 conn(0x7f3a540fd8b0 msgr2=0x7f3a540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1964809927 shutdown_connections 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.786+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1964809927 wait complete. 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 Processor -- start 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 -- start start 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 0x7f3a541988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54111e10 con 0x7f3a54106f00 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54111c90 con 0x7f3a54101dd0 2024-09-05T01:15:42.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5bf43640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54111f90 con 0x7f3a54102780 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59166/0 (socket says 172.21.15.129:59166) 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45950/0 (socket says 172.21.15.129:45950) 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1403589870 learned_addr learned my addr 172.21.15.129:0/1403589870 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a594de640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 0x7f3a541988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 -- 172.21.15.129:0/1403589870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 msgr2=0x7f3a541988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 0x7f3a541988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 -- 172.21.15.129:0/1403589870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 msgr2=0x7f3a54198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a5a4e0640 1 -- 172.21.15.129:0/1403589870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5419d340 con 0x7f3a54106f00 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.787+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:42.792 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a5a4e0640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fd0 tx=0x7f3a4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a4affd640 1 -- 172.21.15.129:0/1403589870 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00ce20 con 0x7f3a54106f00 2024-09-05T01:15:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a4affd640 1 -- 172.21.15.129:0/1403589870 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4c005020 con 0x7f3a54106f00 2024-09-05T01:15:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5419d630 con 0x7f3a54106f00 2024-09-05T01:15:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a4affd640 1 -- 172.21.15.129:0/1403589870 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c010690 con 0x7f3a54106f00 2024-09-05T01:15:42.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.788+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a541a4f10 con 0x7f3a54106f00 2024-09-05T01:15:42.795 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.790+0000 7f3a4affd640 1 -- 172.21.15.129:0/1403589870 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f3a4c0051d0 con 0x7f3a54106f00 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 Processor -- start 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 -- start start 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f6fcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 0x55960f70ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 0x55960f70e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55960f663090 con 0x55960f70bfc0 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960f60aa70 con 0x55960f6fc8b0 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55960f65f050 con 0x55960f6fd1d0 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.791+0000 7f3a4affd640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 0x7f3a400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f6fcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.796 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 0x55960f70e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 0x55960f70ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 0x55960f70e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45952/0 (socket says 172.21.15.129:45952) 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 -- 172.21.15.129:0/2238528875 learned_addr learned my addr 172.21.15.129:0/2238528875 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a5a4e0640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 0x7f3a400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a4affd640 1 -- 172.21.15.129:0/1403589870 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3a4c0cfc50 con 0x7f3a54106f00 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 -- 172.21.15.129:0/2238528875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 msgr2=0x55960f70ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 0x55960f70ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 -- 172.21.15.129:0/2238528875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 msgr2=0x55960f70e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 0x55960f70e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a594de640 1 -- 172.21.15.129:0/2238528875 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960f617130 con 0x55960f6fc8b0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.792+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 0x55960f70ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.793+0000 7f3a594de640 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f6fcc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a440149f0 tx=0x7f3a44005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f99a6b742f0123 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.793+0000 7f3a3ffff640 1 -- 172.21.15.129:0/2238528875 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a440142d0 con 0x55960f6fc8b0 2024-09-05T01:15:42.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.793+0000 7f3a5a4e0640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 0x7f3a400a5150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c09bc40 tx=0x7f3a4c097b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.793+0000 7f3a3ffff640 1 -- 172.21.15.129:0/2238528875 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a440054d0 con 0x55960f6fc8b0 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.793+0000 7f3a3ffff640 1 -- 172.21.15.129:0/2238528875 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400c6d0 con 0x55960f6fc8b0 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2238528875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 msgr2=0x55960f6fcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f6fcc90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3a440149f0 tx=0x7f3a44005850 comp rx=0 tx=0).stop 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2238528875 shutdown_connections 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f70bfc0 0x55960f70e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f6fd1d0 0x55960f70ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2238528875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f6fcc90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.794+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2238528875 >> 172.21.15.129:0/2238528875 conn(0x55960f6f8f60 msgr2=0x55960f6f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2238528875 shutdown_connections 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2238528875 wait complete. 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 -- start start 2024-09-05T01:15:42.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 Processor -- start 2024-09-05T01:15:42.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.795+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f708af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f709030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55960f70eb90 con 0x55960f6fd1d0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960f61df20 con 0x55960f6fc8b0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55960f70ed10 con 0x55960f70bfc0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a5e31c740 10 client.0 _ll_register_callbacks cb 0x55960f5e7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a594de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a594de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59192/0 (socket says 172.21.15.129:59192) 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a594de640 1 -- 172.21.15.129:0/1536161960 learned_addr learned my addr 172.21.15.129:0/1536161960 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f708af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a594de640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f709030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1536161960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 msgr2=0x55960f709030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f709030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 msgr2=0x55960f70b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1536161960 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960f3d42b0 con 0x55960f6fd1d0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a594de640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f708af0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004770 tx=0x7f3a500317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266dccd0cb8c907c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.796+0000 7f3a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50035040 con 0x55960f6fd1d0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3a50004450 con 0x55960f6fd1d0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a440054d0 con 0x55960f6fd1d0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a50005020 con 0x55960f6fd1d0 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a5e31c740 10 client.15366 fetch_fsmap learned FSMap version 27 2024-09-05T01:15:42.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55960f71a430 con 0x55960f6fd1d0 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a440054d0 con 0x55960f6fd1d0 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.797+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3a500035f0 con 0x55960f6fd1d0 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 10 client.15366 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 10 client.15366 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 20 client.15366 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f71ca70 0x55960f71dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 4 client.15366 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55960f670600 con 0x55960f71ca70 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a594de640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f71ca70 0x55960f71dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a594de640 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f71ca70 0x55960f71dee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdf367a416684526 server_cookie=757e277b04ad1b51 in_seq=0 out_seq=0 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.798+0000 7f3a3d7fa640 10 client.15366 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1536161960 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55960f670600 con 0x55960f71ca70 2024-09-05T01:15:42.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a3d7fa640 10 client.15366 handle_command_reply: tid=0 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 client.15366 shutdown 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 msgr2=0x55960f708af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f708af0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3a50004770 tx=0x7f3a500317a0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 shutdown_connections 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f71ca70 0x55960f71dee0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f709030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f708af0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1536161960 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f70b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a3d7fa640 0 client.15366 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.799+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 >> 172.21.15.129:0/1536161960 conn(0x55960f6f8f60 msgr2=0x55960f6fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.800+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 shutdown_connections 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.800+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1536161960 wait complete. 2024-09-05T01:15:42.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.800+0000 7f3a5e31c740 20 client.15366 trim_cache size 0 max 16384 2024-09-05T01:15:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.821+0000 7f3a5e31c740 1 Processor -- start 2024-09-05T01:15:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.821+0000 7f3a5e31c740 1 -- start start 2024-09-05T01:15:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f71dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f71c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55960f71c220 con 0x55960f6fd1d0 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960f71c8f0 con 0x55960f6fc8b0 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55960f71cfc0 con 0x55960f70bfc0 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f71dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59208/0 (socket says 172.21.15.129:59208) 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 -- 172.21.15.129:0/2043388966 learned_addr learned my addr 172.21.15.129:0/2043388966 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f71dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45976/0 (socket says 172.21.15.129:45976) 2024-09-05T01:15:42.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f71c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 -- 172.21.15.129:0/2043388966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 msgr2=0x55960f71c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f71c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 -- 172.21.15.129:0/2043388966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 msgr2=0x55960f71dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f71dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 -- 172.21.15.129:0/2043388966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55960f70bb10 con 0x55960f6fc8b0 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.822+0000 7f3a594de640 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005ce0 tx=0x7f3a44023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8beee4f1b2381d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.823+0000 7f3a3f7fe640 1 -- 172.21.15.129:0/2043388966 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400c6d0 con 0x55960f6fc8b0 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.823+0000 7f3a3f7fe640 1 -- 172.21.15.129:0/2043388966 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4400c870 con 0x55960f6fc8b0 2024-09-05T01:15:42.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.823+0000 7f3a3f7fe640 1 -- 172.21.15.129:0/2043388966 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a44005010 con 0x55960f6fc8b0 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2043388966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 msgr2=0x55960f71d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005ce0 tx=0x7f3a44023040 comp rx=0 tx=0).stop 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2043388966 shutdown_connections 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f71c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fd1d0 0x55960f71dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/2043388966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fc8b0 0x55960f71d810 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2043388966 >> 172.21.15.129:0/2043388966 conn(0x55960f6f8f60 msgr2=0x55960f6f3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2043388966 shutdown_connections 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- 172.21.15.129:0/2043388966 wait complete. 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 -- start start 2024-09-05T01:15:42.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.824+0000 7f3a5e31c740 1 Processor -- start 2024-09-05T01:15:42.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fc8b0 0x55960f7064d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f7052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55960f71dd60 con 0x55960f6fc8b0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55960f71d690 con 0x55960f6fd1d0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55960f71e430 con 0x55960f70bfc0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a5e31c740 10 client.0 _ll_register_callbacks cb 0x7f3a50005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a594de640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fc8b0 0x55960f7064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59218/0 (socket says 172.21.15.129:59218) 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1578332934 learned_addr learned my addr 172.21.15.129:0/1578332934 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1578332934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 msgr2=0x55960f7052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f7052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1578332934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fc8b0 msgr2=0x55960f7064d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fc8b0 0x55960f7064d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.825+0000 7f3a59cdf640 1 -- 172.21.15.129:0/1578332934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55960f705910 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a59cdf640 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a50003bb0 tx=0x7f3a50005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd4e598d907d008b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a50035040 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a4400c870 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3a50004450 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3a50004e70 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a5e31c740 10 client.24962 fetch_fsmap learned FSMap version 27 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55960f726040 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4400c870 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3a500049a0 con 0x55960f6fd1d0 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.826+0000 7f3a5e31c740 10 client.24962 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a5e31c740 10 client.24962 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T01:15:42.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a5e31c740 20 client.24962 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f78f660 0x55960f78e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a5e31c740 4 client.24962 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "status"}) -- 0x55960f66ab10 con 0x55960f78f660 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a594de640 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f78f660 0x55960f78e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a594de640 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f78f660 0x55960f78e340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fd41dd7be9ae6d server_cookie=bdf581ff8a80640b in_seq=0 out_seq=0 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.827+0000 7f3a3d7fa640 10 client.24962 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a3d7fa640 1 -- 172.21.15.129:0/1578332934 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55960f66ab10 con 0x55960f78f660 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a3d7fa640 10 client.24962 handle_command_reply: tid=0 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 client.24962 shutdown 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 msgr2=0x55960f704d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a50003bb0 tx=0x7f3a50005fd0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 shutdown_connections 2024-09-05T01:15:42.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55960f78f660 0x55960f78e340 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55960f70bfc0 0x55960f7052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55960f6fd1d0 0x55960f704d60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 --2- 172.21.15.129:0/1578332934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55960f6fc8b0 0x55960f7064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 >> 172.21.15.129:0/1578332934 conn(0x55960f6f8f60 msgr2=0x55960f6f77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a3d7fa640 0 client.24962 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.828+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 shutdown_connections 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5e31c740 1 -- 172.21.15.129:0/1578332934 wait complete. 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5e31c740 20 client.24962 trim_cache size 0 max 16384 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:15:42.833 INFO:teuthology.orchestra.run.smithi129.stdout: "cluster_fsid": "c8062bdc-6b22-11ef-bcd6-c7b262605968", 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "whoami": 0, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "id": 24488, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "want_state": "up:active", 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "state": "up:active", 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "fs_name": "cephfs", 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "rank_uptime": 155.94229955, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "mdsmap_epoch": 27, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch": 90, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_epoch_barrier": 90, 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout: "uptime": 166.79535782799999 2024-09-05T01:15:42.834 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:15:42.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 msgr2=0x7f3a400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 0x7f3a400a5150 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c09bc40 tx=0x7f3a4c097b30 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 msgr2=0x7f3a5419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fd0 tx=0x7f3a4c00ef10 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 shutdown_connections 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3a400a2c90 0x7f3a400a5150 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a54106f00 0x7f3a5419cc60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3a54102780 0x7f3a541988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 --2- 172.21.15.129:0/1403589870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a54101dd0 0x7f3a54198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.829+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 >> 172.21.15.129:0/1403589870 conn(0x7f3a540fd8b0 msgr2=0x7f3a540fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.830+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 shutdown_connections 2024-09-05T01:15:42.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:15:42.830+0000 7f3a5bf43640 1 -- 172.21.15.129:0/1403589870 wait complete. 2024-09-05T01:15:42.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:42 smithi129 ceph-mon[30979]: pgmap v362: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 360 KiB/s wr, 9 op/s 2024-09-05T01:15:42.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:42 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:43.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:42 smithi183 ceph-mon[34589]: pgmap v362: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 360 KiB/s wr, 9 op/s 2024-09-05T01:15:43.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:42 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:42 smithi194 ceph-mon[34941]: pgmap v362: 129 pgs: 129 active+clean; 4.8 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 360 KiB/s wr, 9 op/s 2024-09-05T01:15:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:42 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:15:43.506 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-05T01:15:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:44 smithi183 ceph-mon[34589]: pgmap v363: 129 pgs: 129 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.6 MiB/s wr, 43 op/s 2024-09-05T01:15:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:44 smithi129 ceph-mon[30979]: pgmap v363: 129 pgs: 129 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.6 MiB/s wr, 43 op/s 2024-09-05T01:15:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:44 smithi194 ceph-mon[34941]: pgmap v363: 129 pgs: 129 active+clean; 18 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.6 MiB/s wr, 43 op/s 2024-09-05T01:15:47.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:46 smithi183 ceph-mon[34589]: pgmap v364: 129 pgs: 129 active+clean; 25 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.4 MiB/s wr, 53 op/s 2024-09-05T01:15:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:46 smithi129 ceph-mon[30979]: pgmap v364: 129 pgs: 129 active+clean; 25 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.4 MiB/s wr, 53 op/s 2024-09-05T01:15:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:46 smithi194 ceph-mon[34941]: pgmap v364: 129 pgs: 129 active+clean; 25 MiB data, 457 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 2.4 MiB/s wr, 53 op/s 2024-09-05T01:15:49.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:48 smithi183 ceph-mon[34589]: pgmap v365: 129 pgs: 129 active+clean; 29 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.0 MiB/s wr, 76 op/s 2024-09-05T01:15:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:48 smithi129 ceph-mon[30979]: pgmap v365: 129 pgs: 129 active+clean; 29 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.0 MiB/s wr, 76 op/s 2024-09-05T01:15:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:48 smithi194 ceph-mon[34941]: pgmap v365: 129 pgs: 129 active+clean; 29 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 3.0 MiB/s wr, 76 op/s 2024-09-05T01:15:51.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:50 smithi183 ceph-mon[34589]: pgmap v366: 129 pgs: 129 active+clean; 38 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2024-09-05T01:15:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:50 smithi129 ceph-mon[30979]: pgmap v366: 129 pgs: 129 active+clean; 38 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2024-09-05T01:15:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:50 smithi194 ceph-mon[34941]: pgmap v366: 129 pgs: 129 active+clean; 38 MiB data, 481 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2024-09-05T01:15:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 -- 172.21.15.129:0/3111712653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df4100a40 msgr2=0x7f1df4100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/3111712653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df4100a40 0x7f1df4100e40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009900 tx=0x7f1de802a410 comp rx=0 tx=0).stop 2024-09-05T01:15:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 -- 172.21.15.129:0/3111712653 shutdown_connections 2024-09-05T01:15:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/3111712653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df41025f0 0x7f1df4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/3111712653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4101c30 0x7f1df41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/3111712653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df4100a40 0x7f1df4100e40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 -- 172.21.15.129:0/3111712653 >> 172.21.15.129:0/3111712653 conn(0x7f1df40fc1e0 msgr2=0x7f1df40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 -- 172.21.15.129:0/3111712653 shutdown_connections 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.588+0000 7f1dfad8f640 1 -- 172.21.15.129:0/3111712653 wait complete. 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 Processor -- start 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 -- start start 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4100a40 0x7f1df4104b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df4101c30 0x7f1df41031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df410c550 con 0x7f1df41025f0 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df410c3d0 con 0x7f1df4100a40 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df410c6d0 con 0x7f1df4101c30 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45994/0 (socket says 172.21.15.129:45994) 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 -- 172.21.15.129:0/1034251027 learned_addr learned my addr 172.21.15.129:0/1034251027 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4100a40 0x7f1df4104b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df3fff640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df4101c30 0x7f1df41031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 -- 172.21.15.129:0/1034251027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df4101c30 msgr2=0x7f1df41031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df4101c30 0x7f1df41031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 -- 172.21.15.129:0/1034251027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4100a40 msgr2=0x7f1df4104b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4100a40 0x7f1df4104b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 -- 172.21.15.129:0/1034251027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df4103fa0 con 0x7f1df41025f0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df932c640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00eb10 tx=0x7f1dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df1ffb640 1 -- 172.21.15.129:0/1034251027 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec004b20 con 0x7f1df41025f0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df1ffb640 1 -- 172.21.15.129:0/1034251027 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec004cc0 con 0x7f1df41025f0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df41a1e20 con 0x7f1df41025f0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.589+0000 7f1df1ffb640 1 -- 172.21.15.129:0/1034251027 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec010690 con 0x7f1df41025f0 2024-09-05T01:15:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.590+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df41a2230 con 0x7f1df41025f0 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.592+0000 7f1df1ffb640 1 -- 172.21.15.129:0/1034251027 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f1dec005020 con 0x7f1df41025f0 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.592+0000 7f1dfdbff740 1 Processor -- start 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.592+0000 7f1dfdbff740 1 -- start start 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df1ffb640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 0x7f1de40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce352da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce353370 0x5635ce355830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635ce2b12f0 con 0x5635ce34e2e0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635ce259270 con 0x5635ce34d9a0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5635ce2ad2b0 con 0x5635ce353370 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 0x7f1de40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59258/0 (socket says 172.21.15.129:59258) 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 -- 172.21.15.129:0/2580881769 learned_addr learned my addr 172.21.15.129:0/2580881769 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df932c640 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce353370 0x5635ce355830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce352da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df1ffb640 1 -- 172.21.15.129:0/1034251027 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1dec0cfc20 con 0x7f1df41025f0 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 -- 172.21.15.129:0/2580881769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce353370 msgr2=0x5635ce355830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce353370 0x5635ce355830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 -- 172.21.15.129:0/2580881769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 msgr2=0x5635ce352da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce352da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 -- 172.21.15.129:0/2580881769 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ce258340 con 0x5635ce34d9a0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df3fff640 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009fd0 tx=0x7f1de0009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a80d1278443d7841 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 0x7f1de40a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1de8004770 tx=0x7f1de8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.593+0000 7f1ddeffd640 1 -- 172.21.15.129:0/2580881769 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0014110 con 0x5635ce34d9a0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1ddeffd640 1 -- 172.21.15.129:0/2580881769 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de00142b0 con 0x5635ce34d9a0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1ddeffd640 1 -- 172.21.15.129:0/2580881769 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00035f0 con 0x5635ce34d9a0 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 -- 172.21.15.129:0/2580881769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 msgr2=0x5635ce34dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009fd0 tx=0x7f1de0009630 comp rx=0 tx=0).stop 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 -- 172.21.15.129:0/2580881769 shutdown_connections 2024-09-05T01:15:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce353370 0x5635ce355830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce352da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/2580881769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34d9a0 0x5635ce34dda0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 -- 172.21.15.129:0/2580881769 >> 172.21.15.129:0/2580881769 conn(0x5635ce349010 msgr2=0x5635ce349410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 -- 172.21.15.129:0/2580881769 shutdown_connections 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.594+0000 7f1dfdbff740 1 -- 172.21.15.129:0/2580881769 wait complete. 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 -- start start 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 Processor -- start 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce357b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce35f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5635ce3562e0 con 0x5635ce34e2e0 2024-09-05T01:15:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5635ce23a290 con 0x5635ce353370 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5635ce356460 con 0x5635ce34d9a0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1dfdbff740 10 client.0 _ll_register_callbacks cb 0x5635ce26b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46018/0 (socket says 172.21.15.129:46018) 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1df3fff640 1 -- 172.21.15.129:0/646121513 learned_addr learned my addr 172.21.15.129:0/646121513 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1df932c640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce35f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.595+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce357b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 -- 172.21.15.129:0/646121513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 msgr2=0x5635ce357b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce357b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 -- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 msgr2=0x5635ce35f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce35f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 -- 172.21.15.129:0/646121513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635cdfc4210 con 0x5635ce34e2e0 2024-09-05T01:15:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1df3fff640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1de00095d0 tx=0x7f1de0014110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a02ff2b8d893ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0004430 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de00142b0 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1de0005050 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1de0030050 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dfdbff740 10 client.15384 fetch_fsmap learned FSMap version 27 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.596+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635ce090620 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de00142b0 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f1de0014ce0 con 0x5635ce34e2e0 2024-09-05T01:15:51.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 10 client.15384 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 20 client.15384 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce365910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 4 client.15384 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ce2bea20 con 0x5635ce363520 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.597+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce365910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.598+0000 7f1dd3fff640 10 client.15384 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.598+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce365910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdedd3ed1a349b2a server_cookie=4e95176a021559d8 in_seq=0 out_seq=0 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.598+0000 7f1dd3fff640 1 -- 172.21.15.129:0/646121513 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635ce2bea20 con 0x5635ce363520 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.598+0000 7f1dd3fff640 10 client.15384 handle_command_reply: tid=0 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.598+0000 7f1dfdbff740 1 client.15384 shutdown 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 msgr2=0x5635ce358070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1de00095d0 tx=0x7f1de0014110 comp rx=0 tx=0).stop 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 shutdown_connections 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce365910 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce35f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dd3fff640 0 client.15384 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34e2e0 0x5635ce358070 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/646121513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce357b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 >> 172.21.15.129:0/646121513 conn(0x5635ce349010 msgr2=0x5635ce34a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 shutdown_connections 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 1 -- 172.21.15.129:0/646121513 wait complete. 2024-09-05T01:15:51.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.599+0000 7f1dfdbff740 20 client.15384 trim_cache size 0 max 16384 2024-09-05T01:15:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.615+0000 7f1dfdbff740 1 Processor -- start 2024-09-05T01:15:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.615+0000 7f1dfdbff740 1 -- start start 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 0x5635ce34a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce34a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41af320 con 0x5635ce34d9a0 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41acbf0 con 0x5635ce353370 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41aaa40 con 0x5635ce34e2e0 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df8b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 0x5635ce34a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:54070/0 (socket says 172.21.15.129:54070) 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df8b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 0x5635ce34a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46034/0 (socket says 172.21.15.129:46034) 2024-09-05T01:15:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 -- 172.21.15.129:0/3738106288 learned_addr learned my addr 172.21.15.129:0/3738106288 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df932c640 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce34a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 -- 172.21.15.129:0/3738106288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 msgr2=0x5635ce34a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce34a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 -- 172.21.15.129:0/3738106288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 msgr2=0x5635ce34a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 0x5635ce34a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 -- 172.21.15.129:0/3738106288 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ce34e7c0 con 0x5635ce34e2e0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1df3fff640 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1de0014690 tx=0x7f1de0009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc79b9bbe53e4693 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.616+0000 7f1dde7fc640 1 -- 172.21.15.129:0/3738106288 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de003a070 con 0x5635ce34e2e0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dde7fc640 1 -- 172.21.15.129:0/3738106288 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de000f8a0 con 0x5635ce34e2e0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dde7fc640 1 -- 172.21.15.129:0/3738106288 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de0004230 con 0x5635ce34e2e0 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 -- 172.21.15.129:0/3738106288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 msgr2=0x5635ce35f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1de0014690 tx=0x7f1de0009530 comp rx=0 tx=0).stop 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 -- 172.21.15.129:0/3738106288 shutdown_connections 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce353370 0x5635ce34a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34e2e0 0x5635ce35f070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/3738106288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce34d9a0 0x5635ce34a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 -- 172.21.15.129:0/3738106288 >> 172.21.15.129:0/3738106288 conn(0x5635ce349010 msgr2=0x5635ce3481c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 -- 172.21.15.129:0/3738106288 shutdown_connections 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.617+0000 7f1dfdbff740 1 -- 172.21.15.129:0/3738106288 wait complete. 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 -- start start 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 Processor -- start 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34e2e0 0x5635ce35d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41ab0b0 con 0x5635ce353370 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41ac520 con 0x5635ce34e2e0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df41ab780 con 0x5635ce34d9a0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1dfdbff740 10 client.0 _ll_register_callbacks cb 0x7f1de0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df932c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df8b2b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34e2e0 0x5635ce35d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df8b2b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:54086/0 (socket says 172.21.15.129:54086) 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df932c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46050/0 (socket says 172.21.15.129:46050) 2024-09-05T01:15:51.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.618+0000 7f1df8b2b640 1 -- 172.21.15.129:0/211907450 learned_addr learned my addr 172.21.15.129:0/211907450 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 -- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34e2e0 msgr2=0x5635ce35d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34e2e0 0x5635ce35d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 -- 172.21.15.129:0/211907450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 msgr2=0x5635ce35e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 -- 172.21.15.129:0/211907450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ce35e620 con 0x5635ce34d9a0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df932c640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1df8b2b640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009900 tx=0x7f1de8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f68f0bb67863ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8035050 con 0x5635ce34d9a0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1de000f8a0 con 0x5635ce34d9a0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1de80035f0 con 0x5635ce34d9a0 2024-09-05T01:15:51.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1de8064050 con 0x5635ce34d9a0 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dfdbff740 10 client.24949 fetch_fsmap learned FSMap version 27 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.619+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5635ce3ce640 con 0x5635ce34d9a0 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de000f8a0 con 0x5635ce34d9a0 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f1de8052800 con 0x5635ce34d9a0 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 10 client.24949 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 10 client.24949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 20 client.24949 populate_metadata read hostname 'smithi129' 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce3d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 4 client.24949 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635ce2b8d70 con 0x5635ce363520 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.620+0000 7f1df3fff640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce3d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:15:51.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.621+0000 7f1df3fff640 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce3d1040 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a930c9bcdf68a8a server_cookie=81b1ff5e13233bc0 in_seq=0 out_seq=0 2024-09-05T01:15:51.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.621+0000 7f1dd3fff640 10 client.24949 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dd3fff640 1 -- 172.21.15.129:0/211907450 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635ce2b8d70 con 0x5635ce363520 2024-09-05T01:15:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dd3fff640 10 client.24949 handle_command_reply: tid=0 2024-09-05T01:15:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 client.24949 shutdown 2024-09-05T01:15:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 msgr2=0x5635ce35cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009900 tx=0x7f1de8009260 comp rx=0 tx=0).stop 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 shutdown_connections 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5635ce363520 0x5635ce3d1040 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dd3fff640 0 client.24949 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5635ce353370 0x5635ce35e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5635ce34e2e0 0x5635ce35d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 --2- 172.21.15.129:0/211907450 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5635ce34d9a0 0x5635ce35cac0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 >> 172.21.15.129:0/211907450 conn(0x5635ce349010 msgr2=0x5635ce3493f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 shutdown_connections 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 1 -- 172.21.15.129:0/211907450 wait complete. 2024-09-05T01:15:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.950+0000 7f1dfdbff740 20 client.24949 trim_cache size 0 max 16384 2024-09-05T01:15:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:15:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:15:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:15:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:15:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 msgr2=0x7f1de40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 0x7f1de40a5190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1de8004770 tx=0x7f1de8000bc0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 msgr2=0x7f1df4103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00eb10 tx=0x7f1dec00c2d0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 shutdown_connections 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de40a2cd0 0x7f1de40a5190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df41025f0 0x7f1df4103710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df4101c30 0x7f1df41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 --2- 172.21.15.129:0/1034251027 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df4100a40 0x7f1df4104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 >> 172.21.15.129:0/1034251027 conn(0x7f1df40fc1e0 msgr2=0x7f1df40698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 shutdown_connections 2024-09-05T01:15:51.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:15:51.951+0000 7f1dfad8f640 1 -- 172.21.15.129:0/1034251027 wait complete. 2024-09-05T01:15:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:52 smithi183 ceph-mon[34589]: pgmap v367: 129 pgs: 129 active+clean; 38 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 101 op/s 2024-09-05T01:15:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:52 smithi129 ceph-mon[30979]: pgmap v367: 129 pgs: 129 active+clean; 38 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 101 op/s 2024-09-05T01:15:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:52 smithi194 ceph-mon[34941]: pgmap v367: 129 pgs: 129 active+clean; 38 MiB data, 485 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 4.2 MiB/s wr, 101 op/s 2024-09-05T01:15:54.413 INFO:teuthology.orchestra.run.smithi129.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-05T01:15:54.417 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-05T01:15:54.683 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/: 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:total 2 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 . 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 data 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data: 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:total 45 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 . 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 5 01:15 base 2024-09-05T01:15:54.684 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Sep 5 01:15 global 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 log 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_commit_ts 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_dynshmem 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 5 01:15 pg_hba.conf 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 5 01:15 pg_ident.conf 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:15 pg_logical 2024-09-05T01:15:54.685 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 pg_multixact 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_notify 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_replslot 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_serial 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_snapshots 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_stat 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_stat_tmp 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 pg_subtrans 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_tblspc 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_twophase 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:15:54.686 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:15 pg_wal 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 pg_xact 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 5 01:15 postgresql.auto.conf 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 5 01:15 postgresql.conf 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base: 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:total 3 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 5 01:15 . 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 1 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 13493 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 13494 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-05T01:15:54.687 INFO:teuthology.orchestra.run.smithi129.stdout:total 7778 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 . 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 5 01:15 .. 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 1247 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1247_vm 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:15 1249 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1249_fsm 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1249_vm 2024-09-05T01:15:54.688 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:15 1259 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1259_vm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:15:54.689 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:15:54.690 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2579 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:15:54.691 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:15:54.692 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606_vm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:15 2608 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2608_vm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:15:54.693 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2610 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2610_vm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:15:54.694 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 5 01:15 2619 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2619_fsm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2619_vm 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:15:54.695 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:15 2658 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 2659 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2662 2024-09-05T01:15:54.696 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2663 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2664 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2665 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2666 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2667 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:15 2673 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:15 2674 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2678 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2679 2024-09-05T01:15:54.697 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:15:54.698 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2696 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2703 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2704 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:15:54.699 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:15:54.700 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:15:54.701 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:15:54.702 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3455 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:15:54.703 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:15:54.704 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:15:54.705 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:15:54.706 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:15:54.707 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:15:54.708 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:15:54.708 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:15:54.708 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:15:54.708 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:15:54.901 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:15:54.902 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:15:54.903 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:15:54.904 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:total 7778 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 . 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 5 01:15 .. 2024-09-05T01:15:54.905 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 1247 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1247_vm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:15 1249 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1249_fsm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1249_vm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:15 1259 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:15:54.906 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1259_vm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:15:54.907 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:15:54.908 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2579 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:15:54.909 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606_vm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:15:54.910 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:15 2608 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2608_vm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2610 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2610_vm 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:15:54.911 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:15:54.912 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 5 01:15 2619 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2619_fsm 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2619_vm 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:15:54.913 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:15 2658 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 2659 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2662 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2663 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2664 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2665 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2666 2024-09-05T01:15:54.914 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2667 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:15 2673 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:15 2674 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2678 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2679 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:15:54.915 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2696 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:15:54.916 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2703 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2704 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:15:54.917 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:15:54.918 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:15:54.919 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:15:54.920 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3455 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:15:54.921 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:15:54.922 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:15:54.923 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:15:54.924 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:15:54.925 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:15:54.926 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:15:55.117 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:15:55.118 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:total 7778 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 . 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 23889417 Sep 5 01:15 .. 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:15:55.119 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 1247 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1247_vm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:15 1249 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1249_fsm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1249_vm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:15 1259 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:15:55.120 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1259_vm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:15:55.121 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:15:55.122 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2579 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:15:55.123 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606_vm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:15 2608 2024-09-05T01:15:55.124 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2608_vm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2610 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2610_vm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:15:55.125 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 5 01:15 2619 2024-09-05T01:15:55.126 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2619_fsm 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2619_vm 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:15 2658 2024-09-05T01:15:55.127 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 2659 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2662 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2663 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2664 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2665 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2666 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2667 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:15 2673 2024-09-05T01:15:55.128 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:15 2674 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2678 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2679 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:15:55.129 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2696 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2703 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2704 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:15:55.130 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:15:55.131 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:15:55.132 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:15:55.133 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3455 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:15:55.134 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:15:55.135 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:15:55.136 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:15:55.137 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:15:55.138 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:15:55.139 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:15:55.140 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:15:55.140 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:15:55.140 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:15:55.156 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:15:55.157 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/global: 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:total 538 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 549376 Sep 5 01:15 . 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1213 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1213_fsm 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1213_vm 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1214 2024-09-05T01:15:55.158 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1214_fsm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1214_vm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 1232 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 1233 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1260 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1260_fsm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1260_vm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1261 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1261_fsm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1261_vm 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1262 2024-09-05T01:15:55.159 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1262_fsm 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1262_vm 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2396 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2396_fsm 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2396_vm 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2397 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2671 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2672 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2676 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2677 2024-09-05T01:15:55.160 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2694 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2695 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2697 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2698 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2846 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2847 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2964 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2965 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2966 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2967 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3592 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3593 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4060 2024-09-05T01:15:55.161 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4061 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4175 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4176 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4177 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4178 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4181 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4182 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4183 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4184 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4185 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4186 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6000 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6001 2024-09-05T01:15:55.162 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6002 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6100 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6114 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6115 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 pg_control 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/log: 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.163 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.163 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:54 smithi183 ceph-mon[34589]: pgmap v368: 129 pgs: 129 active+clean; 43 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.201 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:total 3 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:15 . 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 mappings 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:15 replorigin_checkpoint 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 snapshots 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-05T01:15:55.202 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:15 .. 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:15 .. 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:total 2 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 . 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.203 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 members 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 offsets 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 .. 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 0000 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 .. 2024-09-05T01:15:55.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 0000 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-05T01:15:55.205 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.206 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-05T01:15:55.207 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.207 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.207 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.207 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.207 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-05T01:15:55.214 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 0000 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-05T01:15:55.215 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:total 16386 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:15 . 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:15 000000010000000000000001 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 archive_status 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-05T01:15:55.216 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:15:55.217 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:15:55.217 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:15 .. 2024-09-05T01:15:55.217 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:15:55.218 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-05T01:15:55.218 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:15:55.218 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:15:55.218 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 41283102 Sep 5 01:15 .. 2024-09-05T01:15:55.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 0000 2024-09-05T01:15:55.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:54 smithi129 ceph-mon[30979]: pgmap v368: 129 pgs: 129 active+clean; 43 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-09-05T01:15:55.219 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-05T01:15:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:54 smithi194 ceph-mon[34941]: pgmap v368: 129 pgs: 129 active+clean; 43 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 4.3 MiB/s wr, 279 op/s 2024-09-05T01:15:55.395 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-05T01:15:55.466 INFO:teuthology.orchestra.run.smithi129.stderr:dropping old tables... 2024-09-05T01:15:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-05T01:15:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-05T01:15:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-05T01:15:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-05T01:15:55.472 INFO:teuthology.orchestra.run.smithi129.stderr:creating tables... 2024-09-05T01:15:55.518 INFO:teuthology.orchestra.run.smithi129.stderr:generating data (client-side)... 2024-09-05T01:15:55.951 INFO:teuthology.orchestra.run.smithi129.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.41 s, remaining 12.75 s) 2024-09-05T01:15:56.954 INFO:teuthology.orchestra.run.smithi129.stderr:200000 of 3200000 tuples (6%) done (elapsed 1.41 s, remaining 21.21 s) 2024-09-05T01:15:57.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:56 smithi183 ceph-mon[34589]: pgmap v369: 129 pgs: 129 active+clean; 48 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-09-05T01:15:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:56 smithi129 ceph-mon[30979]: pgmap v369: 129 pgs: 129 active+clean; 48 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-09-05T01:15:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:56 smithi194 ceph-mon[34941]: pgmap v369: 129 pgs: 129 active+clean; 48 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-09-05T01:15:58.071 INFO:teuthology.orchestra.run.smithi129.stderr:300000 of 3200000 tuples (9%) done (elapsed 2.53 s, remaining 24.47 s) 2024-09-05T01:15:58.508 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.2 2024-09-05T01:15:58.508 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:15:58.790 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:15:58.808 INFO:teuthology.orchestra.run.smithi129.stderr:400000 of 3200000 tuples (12%) done (elapsed 3.27 s, remaining 22.88 s) 2024-09-05T01:15:59.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:15:58 smithi183 ceph-mon[34589]: pgmap v370: 129 pgs: 129 active+clean; 61 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-09-05T01:15:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:15:58 smithi129 ceph-mon[30979]: pgmap v370: 129 pgs: 129 active+clean; 61 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-09-05T01:15:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:15:58 smithi194 ceph-mon[34941]: pgmap v370: 129 pgs: 129 active+clean; 61 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-09-05T01:15:59.845 INFO:teuthology.orchestra.run.smithi129.stderr:500000 of 3200000 tuples (15%) done (elapsed 4.30 s, remaining 23.23 s) 2024-09-05T01:16:00.738 INFO:teuthology.orchestra.run.smithi129.stderr:600000 of 3200000 tuples (18%) done (elapsed 5.20 s, remaining 22.53 s) 2024-09-05T01:16:00.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:00 smithi129 ceph-mon[30979]: pgmap v371: 129 pgs: 129 active+clean; 101 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 9.5 MiB/s wr, 356 op/s 2024-09-05T01:16:01.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:00 smithi183 ceph-mon[34589]: pgmap v371: 129 pgs: 129 active+clean; 101 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 9.5 MiB/s wr, 356 op/s 2024-09-05T01:16:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:00 smithi194 ceph-mon[34941]: pgmap v371: 129 pgs: 129 active+clean; 101 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 9.5 MiB/s wr, 356 op/s 2024-09-05T01:16:01.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 -- 172.21.15.129:0/3054499096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 msgr2=0x7f963c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:01.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 --2- 172.21.15.129:0/3054499096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c10ec40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9634009cd0 tx=0x7f963402a8e0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 -- 172.21.15.129:0/3054499096 shutdown_connections 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 --2- 172.21.15.129:0/3054499096 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c10ec40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 --2- 172.21.15.129:0/3054499096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 0x7f963c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 --2- 172.21.15.129:0/3054499096 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 0x7f963c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 -- 172.21.15.129:0/3054499096 >> 172.21.15.129:0/3054499096 conn(0x7f963c0fc7d0 msgr2=0x7f963c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 -- 172.21.15.129:0/3054499096 shutdown_connections 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.401+0000 7f9642c8a640 1 -- 172.21.15.129:0/3054499096 wait complete. 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.402+0000 7f9642c8a640 1 Processor -- start 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.402+0000 7f9642c8a640 1 -- start start 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.402+0000 7f9642c8a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 0x7f963c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.402+0000 7f9642c8a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 0x7f963c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.402+0000 7f9642c8a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:01.415 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9642c8a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f963c111fa0 con 0x7f963c1082a0 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9642c8a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f963c111e20 con 0x7f963c1036e0 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9642c8a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f963c112120 con 0x7f963c102d30 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9640a26640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 0x7f963c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49684/0 (socket says 172.21.15.129:49684) 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 -- 172.21.15.129:0/667226965 learned_addr learned my addr 172.21.15.129:0/667226965 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f963bfff640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 0x7f963c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 -- 172.21.15.129:0/667226965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 msgr2=0x7f963c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 0x7f963c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 -- 172.21.15.129:0/667226965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 msgr2=0x7f963c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:01.416 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 0x7f963c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 -- 172.21.15.129:0/667226965 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c19d3f0 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9641227640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9634005e30 tx=0x7f9634003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634004240 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9634031ce0 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.403+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c19d680 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.404+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9634030ea0 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.404+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c1a4fc0 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.405+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f963c103110 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.406+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f96340305a0 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.406+0000 7f9639ffb640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 0x7f962c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.407+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f963402e030 con 0x7f963c1082a0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.407+0000 7f9640a26640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 0x7f962c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:01.417 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.407+0000 7f9640a26640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 0x7f962c0a5150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9630005fd0 tx=0x7f963000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:01.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.411+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f963404b060 con 0x7f963c1082a0 2024-09-05T01:16:01.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.674+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f963c10a2b0 con 0x7f963c1082a0 2024-09-05T01:16:01.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.675+0000 7f9639ffb640 1 -- 172.21.15.129:0/667226965 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f96340bd660 con 0x7f963c1082a0 2024-09-05T01:16:01.678 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:16:01.678 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"btime":"2024-09-05T01:15:41:859752+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:41.039933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:16:01.678 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 27 2024-09-05T01:16:01.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.678+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 msgr2=0x7f962c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:01.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.678+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 0x7f962c0a5150 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9630005fd0 tx=0x7f963000b390 comp rx=0 tx=0).stop 2024-09-05T01:16:01.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.678+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 msgr2=0x7f963c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.678+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9634005e30 tx=0x7f9634003ba0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 shutdown_connections 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f962c0a2c90 0x7f962c0a5150 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f963c1082a0 0x7f963c19cd10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f963c1036e0 0x7f963c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 --2- 172.21.15.129:0/667226965 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f963c102d30 0x7f963c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 >> 172.21.15.129:0/667226965 conn(0x7f963c0fc7d0 msgr2=0x7f963c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 shutdown_connections 2024-09-05T01:16:01.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:01.679+0000 7f9642c8a640 1 -- 172.21.15.129:0/667226965 wait complete. 2024-09-05T01:16:01.953 INFO:teuthology.orchestra.run.smithi129.stderr:700000 of 3200000 tuples (21%) done (elapsed 6.41 s, remaining 22.91 s) 2024-09-05T01:16:01.991 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:01 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/667226965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:02.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:01 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/667226965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:02.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:01 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/667226965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:02.456 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0' 2024-09-05T01:16:02.456 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 148.2 --await-for 45.0 2024-09-05T01:16:02.745 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:16:03.035 INFO:teuthology.orchestra.run.smithi129.stderr:800000 of 3200000 tuples (25%) done (elapsed 7.50 s, remaining 22.49 s) 2024-09-05T01:16:03.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:02 smithi183 ceph-mon[34589]: pgmap v372: 129 pgs: 129 active+clean; 122 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 11 MiB/s wr, 339 op/s 2024-09-05T01:16:03.177 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:02 smithi129 ceph-mon[30979]: pgmap v372: 129 pgs: 129 active+clean; 122 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 11 MiB/s wr, 339 op/s 2024-09-05T01:16:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:02 smithi194 ceph-mon[34941]: pgmap v372: 129 pgs: 129 active+clean; 122 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 11 MiB/s wr, 339 op/s 2024-09-05T01:16:03.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- 172.21.15.129:0/779204981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3278102dd0 msgr2=0x7f32781031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 --2- 172.21.15.129:0/779204981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3278102dd0 0x7f32781031b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3268009900 tx=0x7f326802a480 comp rx=0 tx=0).stop 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- 172.21.15.129:0/779204981 shutdown_connections 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 --2- 172.21.15.129:0/779204981 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 --2- 172.21.15.129:0/779204981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32781036f0 0x7f327810bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 --2- 172.21.15.129:0/779204981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3278102dd0 0x7f32781031b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- 172.21.15.129:0/779204981 >> 172.21.15.129:0/779204981 conn(0x7f32780fc790 msgr2=0x7f32780febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- 172.21.15.129:0/779204981 shutdown_connections 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- 172.21.15.129:0/779204981 wait complete. 2024-09-05T01:16:03.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 Processor -- start 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.288+0000 7f327e54e640 1 -- start start 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3278111e10 con 0x7f32781036f0 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3278111c90 con 0x7f3278102dd0 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327e54e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3278111f90 con 0x7f327810c170 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f3277fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f3277fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:57622/0 (socket says 172.21.15.129:57622) 2024-09-05T01:16:03.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46396/0 (socket says 172.21.15.129:46396) 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f3277fff640 1 -- 172.21.15.129:0/3034010434 learned_addr learned my addr 172.21.15.129:0/3034010434 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327caeb640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 -- 172.21.15.129:0/3034010434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 msgr2=0x7f327819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 -- 172.21.15.129:0/3034010434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 msgr2=0x7f32781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 -- 172.21.15.129:0/3034010434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327819d380 con 0x7f32781036f0 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f327caeb640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f3277fff640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f326ffff640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f326000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.289+0000 7f3275ffb640 1 -- 172.21.15.129:0/3034010434 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260009c30 con 0x7f32781036f0 2024-09-05T01:16:03.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.290+0000 7f3275ffb640 1 -- 172.21.15.129:0/3034010434 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3260009dd0 con 0x7f32781036f0 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.290+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327819d670 con 0x7f32781036f0 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.290+0000 7f3275ffb640 1 -- 172.21.15.129:0/3034010434 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260003e20 con 0x7f32781036f0 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.290+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32781a4f50 con 0x7f32781036f0 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.292+0000 7f3275ffb640 1 -- 172.21.15.129:0/3034010434 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f32600035f0 con 0x7f32781036f0 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.292+0000 7f3280927740 1 Processor -- start 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 -- start start 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3275ffb640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 0x7f32640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c4cf492a0 con 0x559c4cfe3760 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c4cef0b10 con 0x559c4cfee940 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3280927740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559c4cf45260 con 0x559c4cfe4080 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3275ffb640 1 -- 172.21.15.129:0/3034010434 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f32600cf0f0 con 0x7f32781036f0 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f327caeb640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 0x7f32640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46404/0 (socket says 172.21.15.129:46404) 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 -- 172.21.15.129:0/3348589310 learned_addr learned my addr 172.21.15.129:0/3348589310 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f326ffff640 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 -- 172.21.15.129:0/3348589310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 msgr2=0x559c4cfee400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfee400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 -- 172.21.15.129:0/3348589310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 msgr2=0x559c4cff0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.293+0000 7f3277fff640 1 -- 172.21.15.129:0/3348589310 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4ccbab90 con 0x559c4cfe3760 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.294+0000 7f327caeb640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 0x7f32640a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3270005fd0 tx=0x7f32700107a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.294+0000 7f3277fff640 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3268000c00 tx=0x7f3268032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36151ebd70d6960d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.294+0000 7f326e7fc640 1 -- 172.21.15.129:0/3348589310 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268035040 con 0x559c4cfe3760 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.294+0000 7f326e7fc640 1 -- 172.21.15.129:0/3348589310 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3268031820 con 0x559c4cfe3760 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.294+0000 7f326e7fc640 1 -- 172.21.15.129:0/3348589310 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32680049e0 con 0x559c4cfe3760 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- 172.21.15.129:0/3348589310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 msgr2=0x559c4cfe3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3268000c00 tx=0x7f3268032f60 comp rx=0 tx=0).stop 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- 172.21.15.129:0/3348589310 shutdown_connections 2024-09-05T01:16:03.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfee400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 --2- 172.21.15.129:0/3348589310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfe3b40 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- 172.21.15.129:0/3348589310 >> 172.21.15.129:0/3348589310 conn(0x559c4cfdfcf0 msgr2=0x559c4cfe00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- 172.21.15.129:0/3348589310 shutdown_connections 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- 172.21.15.129:0/3348589310 wait complete. 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.295+0000 7f3280927740 1 -- start start 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 Processor -- start 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfe4080 0x559c4cfebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 0x559c4cfec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559c4cff1510 con 0x559c4cfe3760 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559c4ced16e0 con 0x559c4cfe4080 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559c4cff1690 con 0x559c4cfee940 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46420/0 (socket says 172.21.15.129:46420) 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3277fff640 1 -- 172.21.15.129:0/364504126 learned_addr learned my addr 172.21.15.129:0/364504126 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 0x559c4cfec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3280927740 10 client.0 _ll_register_callbacks cb 0x559c4cef5a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f326ffff640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfe4080 0x559c4cfebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 -- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfe4080 msgr2=0x559c4cfebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfe4080 0x559c4cfebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 -- 172.21.15.129:0/364504126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 msgr2=0x559c4cfeb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f327caeb640 1 -- 172.21.15.129:0/364504126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4cffa980 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.296+0000 7f3277fff640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f327caeb640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 0x559c4cfec1d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3270010be0 tx=0x7f3270004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=932799924d0abe56 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270020450 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f32700205f0 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3268031820 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f32700207f0 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f3280927740 10 client.24967 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c4cffd340 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3268031820 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f32700298f0 con 0x559c4cfee940 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.297+0000 7f3280927740 10 client.24967 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:03.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3280927740 10 client.24967 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3280927740 20 client.24967 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4cffe850 0x559c4d000cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3280927740 4 client.24967 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4cf569d0 con 0x559c4cffe850 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3277fff640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4cffe850 0x559c4d000cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f32577fe640 10 client.24967 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.298+0000 7f3277fff640 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4cffe850 0x559c4d000cc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6fcc1f4fc53dcd5 server_cookie=482916cae28bc84c in_seq=0 out_seq=0 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f32577fe640 1 -- 172.21.15.129:0/364504126 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c4cf569d0 con 0x559c4cffe850 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f32577fe640 10 client.24967 handle_command_reply: tid=0 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 client.24967 shutdown 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 msgr2=0x559c4cfec1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 0x559c4cfec1d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3270010be0 tx=0x7f3270004ab0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 shutdown_connections 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4cffe850 0x559c4d000cc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfee940 0x559c4cfec1d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfe4080 0x559c4cfebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 --2- 172.21.15.129:0/364504126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f32577fe640 0 client.24967 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 >> 172.21.15.129:0/364504126 conn(0x559c4cfdfcf0 msgr2=0x559c4cfdee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 shutdown_connections 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.299+0000 7f3280927740 1 -- 172.21.15.129:0/364504126 wait complete. 2024-09-05T01:16:03.321 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.300+0000 7f3280927740 20 client.24967 trim_cache size 0 max 16384 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.319+0000 7f3280927740 1 Processor -- start 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.319+0000 7f3280927740 1 -- start start 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfdee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4d006590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32781b1680 con 0x559c4cfe3760 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32781aef50 con 0x559c4cfee940 2024-09-05T01:16:03.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3280927740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32781acda0 con 0x559c4cfe4080 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46432/0 (socket says 172.21.15.129:46432) 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3277fff640 1 -- 172.21.15.129:0/2950465662 learned_addr learned my addr 172.21.15.129:0/2950465662 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfdee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f327caeb640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4d006590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 -- 172.21.15.129:0/2950465662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 msgr2=0x559c4d006590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4d006590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 -- 172.21.15.129:0/2950465662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 msgr2=0x559c4cfeb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 -- 172.21.15.129:0/2950465662 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4cfdfb20 con 0x559c4cfe4080 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f3277fff640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.320+0000 7f326ffff640 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfdee70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32600cc690 tx=0x7f32600977b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844ba476f308c2a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.321+0000 7f326dffb640 1 -- 172.21.15.129:0/2950465662 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32600b4070 con 0x559c4cfe4080 2024-09-05T01:16:03.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.321+0000 7f326dffb640 1 -- 172.21.15.129:0/2950465662 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32600cc8e0 con 0x559c4cfe4080 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.321+0000 7f326dffb640 1 -- 172.21.15.129:0/2950465662 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3260020040 con 0x559c4cfe4080 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.321+0000 7f3280927740 1 -- 172.21.15.129:0/2950465662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 msgr2=0x559c4cfdee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfdee70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f32600cc690 tx=0x7f32600977b0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 -- 172.21.15.129:0/2950465662 shutdown_connections 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4d006590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cfdee70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 --2- 172.21.15.129:0/2950465662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cfeb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 -- 172.21.15.129:0/2950465662 >> 172.21.15.129:0/2950465662 conn(0x559c4cfdfcf0 msgr2=0x559c4cfed850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 -- 172.21.15.129:0/2950465662 shutdown_connections 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 -- 172.21.15.129:0/2950465662 wait complete. 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 -- start start 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.322+0000 7f3280927740 1 Processor -- start 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cff2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32781ad410 con 0x559c4cfe3760 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32781ae880 con 0x559c4cfee940 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f32781adae0 con 0x559c4cfe4080 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3280927740 10 client.0 _ll_register_callbacks cb 0x7f3270011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46436/0 (socket says 172.21.15.129:46436) 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 -- 172.21.15.129:0/1786058855 learned_addr learned my addr 172.21.15.129:0/1786058855 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f327caeb640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f326ffff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cff2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 -- 172.21.15.129:0/1786058855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 msgr2=0x559c4cff2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cff2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 -- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 msgr2=0x559c4cff2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 -- 172.21.15.129:0/1786058855 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4cfdb260 con 0x559c4cfe3760 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f327caeb640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f32577fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.323+0000 7f3277fff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3268032ad0 tx=0x7f3268005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1beffbd7e1af47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268057020 con 0x559c4cfe3760 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3268004240 con 0x559c4cfe3760 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32600cc8e0 con 0x559c4cfe3760 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3268035040 con 0x559c4cfe3760 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 10 client.15414 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:03.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559c4d06d5d0 con 0x559c4cfe3760 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32600cc8e0 con 0x559c4cfe3760 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3268052500 con 0x559c4cfe3760 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 10 client.15414 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 10 client.15414 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 20 client.15414 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.324+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4d06d9d0 0x559c4d06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.325+0000 7f3280927740 4 client.15414 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}] 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.325+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.2, "await_for": 45.0}) -- 0x559c4cf50d20 con 0x559c4d06d9d0 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.325+0000 7f326ffff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4d06d9d0 0x559c4d06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.325+0000 7f32577fe640 10 client.15414 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:03.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.325+0000 7f326ffff640 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4d06d9d0 0x559c4d06fdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ff7257555b3ff0 server_cookie=e9376d0846d45dab in_seq=0 out_seq=0 2024-09-05T01:16:03.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f32577fe640 1 -- 172.21.15.129:0/1786058855 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559c4cf50d20 con 0x559c4d06d9d0 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f32577fe640 10 client.15414 handle_command_reply: tid=0 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 client.15414 shutdown 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 msgr2=0x559c4cff6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3268032ad0 tx=0x7f3268005520 comp rx=0 tx=0).stop 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 shutdown_connections 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559c4d06d9d0 0x559c4d06fdc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559c4cfee940 0x559c4cff2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559c4cfe4080 0x559c4cff2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f32577fe640 0 client.15414 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 --2- 172.21.15.129:0/1786058855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559c4cfe3760 0x559c4cff6e10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 >> 172.21.15.129:0/1786058855 conn(0x559c4cfdfcf0 msgr2=0x559c4cfe59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 shutdown_connections 2024-09-05T01:16:03.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 1 -- 172.21.15.129:0/1786058855 wait complete. 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.729+0000 7f3280927740 20 client.15414 trim_cache size 0 max 16384 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 27, 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 2, 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "83ea5152": { 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 2, 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:16:03.735 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 148.2, 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:03.736 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 msgr2=0x7f32640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 0x7f32640a5150 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3270005fd0 tx=0x7f32700107a0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 msgr2=0x7f3278198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3260005fd0 tx=0x7f326000dde0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 shutdown_connections 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32640a2c90 0x7f32640a5150 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f327810c170 0x7f327819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f32781036f0 0x7f3278198910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 --2- 172.21.15.129:0/3034010434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102dd0 0x7f32781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 >> 172.21.15.129:0/3034010434 conn(0x7f32780fc790 msgr2=0x7f32780fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 shutdown_connections 2024-09-05T01:16:03.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:03.730+0000 7f327e54e640 1 -- 172.21.15.129:0/3034010434 wait complete. 2024-09-05T01:16:04.943 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '83ea5152', quiesce took 6.4 seconds. Will release after: 21.799999999999997 2024-09-05T01:16:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- 172.21.15.129:0/698388921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac100a40 msgr2=0x7f28ac100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/698388921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac100a40 0x7f28ac100e40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009900 tx=0x7f28a802a410 comp rx=0 tx=0).stop 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- 172.21.15.129:0/698388921 shutdown_connections 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/698388921 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac1025f0 0x7f28ac108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/698388921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac101c30 0x7f28ac1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/698388921 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac100a40 0x7f28ac100e40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- 172.21.15.129:0/698388921 >> 172.21.15.129:0/698388921 conn(0x7f28ac0fc1e0 msgr2=0x7f28ac0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- 172.21.15.129:0/698388921 shutdown_connections 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- 172.21.15.129:0/698388921 wait complete. 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 Processor -- start 2024-09-05T01:16:05.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.045+0000 7f28b2b6d640 1 -- start start 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 0x7f28ac197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac101c30 0x7f28ac198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10d680 con 0x7f28ac1025f0 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10d500 con 0x7f28ac100a40 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac10d800 con 0x7f28ac101c30 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46452/0 (socket says 172.21.15.129:46452) 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 -- 172.21.15.129:0/4110372374 learned_addr learned my addr 172.21.15.129:0/4110372374 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b0909640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 0x7f28ac197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28a3fff640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac101c30 0x7f28ac198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 -- 172.21.15.129:0/4110372374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac101c30 msgr2=0x7f28ac198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac101c30 0x7f28ac198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 -- 172.21.15.129:0/4110372374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 msgr2=0x7f28ac197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 0x7f28ac197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 -- 172.21.15.129:0/4110372374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28ac19f940 con 0x7f28ac1025f0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b0909640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 0x7f28ac197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b110a640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f28a400eaa0 tx=0x7f28a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28a1ffb640 1 -- 172.21.15.129:0/4110372374 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a400cdb0 con 0x7f28ac1025f0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28a1ffb640 1 -- 172.21.15.129:0/4110372374 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a4005020 con 0x7f28ac1025f0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28ac19fc30 con 0x7f28ac1025f0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.046+0000 7f28a1ffb640 1 -- 172.21.15.129:0/4110372374 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40116d0 con 0x7f28ac1025f0 2024-09-05T01:16:05.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.047+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28ac1a0130 con 0x7f28ac1025f0 2024-09-05T01:16:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28a1ffb640 1 -- 172.21.15.129:0/4110372374 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f28a40051d0 con 0x7f28ac1025f0 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 Processor -- start 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 -- start start 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2f2e0 0x55f311a33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a36830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f3119922f0 con 0x55f311a2e9a0 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f31193a270 con 0x55f311a2f2e0 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.049+0000 7f28b59e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f31198e2b0 con 0x55f311a34370 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46458/0 (socket says 172.21.15.129:46458) 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 -- 172.21.15.129:0/3604323849 learned_addr learned my addr 172.21.15.129:0/3604323849 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:05.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a1ffb640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 0x7f28940a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a36830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28b0909640 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2f2e0 0x55f311a33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 -- 172.21.15.129:0/3604323849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 msgr2=0x55f311a36830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a36830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28b110a640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 0x7f28940a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 -- 172.21.15.129:0/3604323849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2f2e0 msgr2=0x55f311a33da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2f2e0 0x55f311a33da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 -- 172.21.15.129:0/3604323849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f311939340 con 0x55f311a2e9a0 2024-09-05T01:16:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a1ffb640 1 -- 172.21.15.129:0/4110372374 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f28a40cf990 con 0x7f28ac1025f0 2024-09-05T01:16:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f289c013950 tx=0x7f289c00b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e363ae67bd15ac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f2892ffd640 1 -- 172.21.15.129:0/3604323849 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c004230 con 0x55f311a2e9a0 2024-09-05T01:16:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f28b110a640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 0x7f28940a5190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28a4097ff0 tx=0x7f28a406ab10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f2892ffd640 1 -- 172.21.15.129:0/3604323849 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f289c0043d0 con 0x55f311a2e9a0 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.050+0000 7f2892ffd640 1 -- 172.21.15.129:0/3604323849 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c0038e0 con 0x55f311a2e9a0 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- 172.21.15.129:0/3604323849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 msgr2=0x55f311a2eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f289c013950 tx=0x7f289c00b6e0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- 172.21.15.129:0/3604323849 shutdown_connections 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a36830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2f2e0 0x55f311a33da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3604323849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311a2eda0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- 172.21.15.129:0/3604323849 >> 172.21.15.129:0/3604323849 conn(0x55f311a2a010 msgr2=0x55f311a2a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- 172.21.15.129:0/3604323849 shutdown_connections 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- 172.21.15.129:0/3604323849 wait complete. 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.051+0000 7f28b59e6740 1 -- start start 2024-09-05T01:16:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 Processor -- start 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2e9a0 0x55f311a38ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311a40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f311a372e0 con 0x55f311a2f2e0 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f31191b290 con 0x55f311a34370 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f311a37460 con 0x55f311a2e9a0 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b59e6740 10 client.0 _ll_register_callbacks cb 0x55f31194c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46472/0 (socket says 172.21.15.129:46472) 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 -- 172.21.15.129:0/4143891909 learned_addr learned my addr 172.21.15.129:0/4143891909 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28a3fff640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2e9a0 0x55f311a38ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28a3fff640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311a40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 -- 172.21.15.129:0/4143891909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2e9a0 msgr2=0x55f311a38ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2e9a0 0x55f311a38ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 -- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 msgr2=0x55f311a40050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311a40050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.052+0000 7f28b0909640 1 -- 172.21.15.129:0/4143891909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3116a5210 con 0x55f311a2f2e0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b0909640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f28a80314e0 tx=0x7f28a8031800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2a9d9312f57cb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035070 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f28a8004240 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c0043d0 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f28a8004440 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f311771620 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f289c0043d0 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f28a8004bf0 con 0x55f311a2f2e0 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 20 client.15426 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311a447e0 0x55f311a46bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 4 client.15426 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.053+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f31199fa20 con 0x55f311a447e0 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.054+0000 7f28a3fff640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311a447e0 0x55f311a46bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.054+0000 7f288bfff640 10 client.15426 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.054+0000 7f28a3fff640 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311a447e0 0x55f311a46bf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748e3585768088e4 server_cookie=feef8ed5475a61da in_seq=0 out_seq=0 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f288bfff640 1 -- 172.21.15.129:0/4143891909 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f31199fa20 con 0x55f311a447e0 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f288bfff640 10 client.15426 handle_command_reply: tid=0 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 client.15426 shutdown 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 msgr2=0x55f311a39010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f28a80314e0 tx=0x7f28a8031800 comp rx=0 tx=0).stop 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 shutdown_connections 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311a447e0 0x55f311a46bf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311a40050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f288bfff640 0 client.15426 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a39010 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 --2- 172.21.15.129:0/4143891909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2e9a0 0x55f311a38ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 >> 172.21.15.129:0/4143891909 conn(0x55f311a2a010 msgr2=0x55f311a2b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 shutdown_connections 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 1 -- 172.21.15.129:0/4143891909 wait complete. 2024-09-05T01:16:05.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.055+0000 7f28b59e6740 20 client.15426 trim_cache size 0 max 16384 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 Processor -- start 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 -- start start 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 0x55f311a2af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1ad050 con 0x55f311a2f2e0 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1aa920 con 0x55f311a2e9a0 2024-09-05T01:16:05.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.069+0000 7f28b59e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a8770 con 0x55f311a34370 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46474/0 (socket says 172.21.15.129:46474) 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 -- 172.21.15.129:0/3666305378 learned_addr learned my addr 172.21.15.129:0/3666305378 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 0x55f311a2af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 -- 172.21.15.129:0/3666305378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 msgr2=0x55f311a2b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a2b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 -- 172.21.15.129:0/3666305378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 msgr2=0x55f311a2af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 0x55f311a2af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 -- 172.21.15.129:0/3666305378 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f311a47270 con 0x55f311a2f2e0 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28a3fff640 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 0x55f311a2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28b0909640 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f28a802a930 tx=0x7f28a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc4a6546ff7d1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28927fc640 1 -- 172.21.15.129:0/3666305378 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8056070 con 0x55f311a2f2e0 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28927fc640 1 -- 172.21.15.129:0/3666305378 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a8003a30 con 0x55f311a2f2e0 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.070+0000 7f28927fc640 1 -- 172.21.15.129:0/3666305378 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035070 con 0x55f311a2f2e0 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- 172.21.15.129:0/3666305378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 msgr2=0x55f311a2b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f28a802a930 tx=0x7f28a8009250 comp rx=0 tx=0).stop 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- 172.21.15.129:0/3666305378 shutdown_connections 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a34370 0x55f311a2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2f2e0 0x55f311a2b460 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- 172.21.15.129:0/3666305378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a2e9a0 0x55f311a2af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- 172.21.15.129:0/3666305378 >> 172.21.15.129:0/3666305378 conn(0x55f311a2a010 msgr2=0x55f311a38ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- 172.21.15.129:0/3666305378 shutdown_connections 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- 172.21.15.129:0/3666305378 wait complete. 2024-09-05T01:16:05.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- start start 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 Processor -- start 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 0x55f311aab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311ab2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a8de0 con 0x55f311a2e9a0 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1aa250 con 0x55f311a34370 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.071+0000 7f28b59e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f28ac1a94b0 con 0x55f311a2f2e0 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b59e6740 10 client.0 _ll_register_callbacks cb 0x7f28a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46478/0 (socket says 172.21.15.129:46478) 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 -- 172.21.15.129:0/734029711 learned_addr learned my addr 172.21.15.129:0/734029711 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b0909640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 0x55f311aab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 -- 172.21.15.129:0/734029711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 msgr2=0x55f311aab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 0x55f311aab3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 -- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 msgr2=0x55f311ab2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311ab2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 -- 172.21.15.129:0/734029711 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f311ab2a20 con 0x55f311a2e9a0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b0909640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 0x55f311aab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28a3fff640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f289c005f90 tx=0x7f289c005fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7adc9c5067ff223 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f288bfff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c013250 con 0x55f311a2e9a0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f289c02fe80 con 0x55f311a2e9a0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28a8003a30 con 0x55f311a2e9a0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f289c011960 con 0x55f311a2e9a0 2024-09-05T01:16:05.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b59e6740 10 client.15432 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f311ab3ca0 con 0x55f311a2e9a0 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.072+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28a8003a30 con 0x55f311a2e9a0 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f289c004680 con 0x55f311a2e9a0 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 10 client.15432 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 20 client.15432 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311ab4590 0x55f311ab69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 4 client.15432 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f311999d70 con 0x55f311ab4590 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b0909640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311ab4590 0x55f311ab69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f288bfff640 10 client.15432 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:05.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.073+0000 7f28b0909640 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311ab4590 0x55f311ab69b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b2bd5d1a91aede server_cookie=c39b98892a0c926 in_seq=0 out_seq=0 2024-09-05T01:16:05.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f288bfff640 1 -- 172.21.15.129:0/734029711 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f311999d70 con 0x55f311ab4590 2024-09-05T01:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f288bfff640 10 client.15432 handle_command_reply: tid=0 2024-09-05T01:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 client.15432 shutdown 2024-09-05T01:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 msgr2=0x55f311aaae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f289c005f90 tx=0x7f289c005fc0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 shutdown_connections 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f311ab4590 0x55f311ab69b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f311a34370 0x55f311ab2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f288bfff640 0 client.15432 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f311a2f2e0 0x55f311aab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 --2- 172.21.15.129:0/734029711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f311a2e9a0 0x55f311aaae90 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 >> 172.21.15.129:0/734029711 conn(0x55f311a2a010 msgr2=0x55f311a2af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 shutdown_connections 2024-09-05T01:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 1 -- 172.21.15.129:0/734029711 wait complete. 2024-09-05T01:16:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.131+0000 7f28b59e6740 20 client.15432 trim_cache size 0 max 16384 2024-09-05T01:16:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:16:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:16:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:16:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:16:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 msgr2=0x7f28940a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 0x7f28940a5190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f28a4097ff0 tx=0x7f28a406ab10 comp rx=0 tx=0).stop 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 msgr2=0x7f28ac19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f28a400eaa0 tx=0x7f28a400ef60 comp rx=0 tx=0).stop 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 shutdown_connections 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f28940a2cd0 0x7f28940a5190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f28ac1025f0 0x7f28ac19f2a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f28ac101c30 0x7f28ac198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 --2- 172.21.15.129:0/4110372374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f28ac100a40 0x7f28ac197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 >> 172.21.15.129:0/4110372374 conn(0x7f28ac0fc1e0 msgr2=0x7f28ac1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 shutdown_connections 2024-09-05T01:16:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:05.132+0000 7f28b2b6d640 1 -- 172.21.15.129:0/4110372374 wait complete. 2024-09-05T01:16:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:04 smithi183 ceph-mon[34589]: pgmap v373: 129 pgs: 129 active+clean; 198 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 25 MiB/s wr, 346 op/s 2024-09-05T01:16:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:04 smithi129 ceph-mon[30979]: pgmap v373: 129 pgs: 129 active+clean; 198 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 25 MiB/s wr, 346 op/s 2024-09-05T01:16:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:04 smithi194 ceph-mon[34941]: pgmap v373: 129 pgs: 129 active+clean; 198 MiB data, 815 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 25 MiB/s wr, 346 op/s 2024-09-05T01:16:07.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:06 smithi183 ceph-mon[34589]: pgmap v374: 129 pgs: 129 active+clean; 214 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 28 MiB/s wr, 167 op/s 2024-09-05T01:16:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:06 smithi129 ceph-mon[30979]: pgmap v374: 129 pgs: 129 active+clean; 214 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 28 MiB/s wr, 167 op/s 2024-09-05T01:16:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:06 smithi194 ceph-mon[34941]: pgmap v374: 129 pgs: 129 active+clean; 214 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 28 MiB/s wr, 167 op/s 2024-09-05T01:16:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:08 smithi183 ceph-mon[34589]: pgmap v375: 129 pgs: 129 active+clean; 226 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 MiB/s wr, 72 op/s 2024-09-05T01:16:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:16:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:16:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:16:09.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:08 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:16:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:08 smithi129 ceph-mon[30979]: pgmap v375: 129 pgs: 129 active+clean; 226 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 MiB/s wr, 72 op/s 2024-09-05T01:16:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:16:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:16:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:16:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:08 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:16:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:08 smithi194 ceph-mon[34941]: pgmap v375: 129 pgs: 129 active+clean; 226 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 29 MiB/s wr, 72 op/s 2024-09-05T01:16:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:16:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:16:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:16:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:08 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:16:11.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:10 smithi183 ceph-mon[34589]: pgmap v376: 129 pgs: 129 active+clean; 229 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 28 MiB/s wr, 50 op/s 2024-09-05T01:16:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:10 smithi129 ceph-mon[30979]: pgmap v376: 129 pgs: 129 active+clean; 229 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 28 MiB/s wr, 50 op/s 2024-09-05T01:16:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:10 smithi194 ceph-mon[34941]: pgmap v376: 129 pgs: 129 active+clean; 229 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 28 MiB/s wr, 50 op/s 2024-09-05T01:16:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:12 smithi129 ceph-mon[30979]: pgmap v377: 129 pgs: 129 active+clean; 225 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-09-05T01:16:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:12 smithi194 ceph-mon[34941]: pgmap v377: 129 pgs: 129 active+clean; 225 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-09-05T01:16:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:12 smithi183 ceph-mon[34589]: pgmap v377: 129 pgs: 129 active+clean; 225 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 45 op/s 2024-09-05T01:16:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:14 smithi129 ceph-mon[30979]: pgmap v378: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 39 op/s 2024-09-05T01:16:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:14 smithi194 ceph-mon[34941]: pgmap v378: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 39 op/s 2024-09-05T01:16:15.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:14 smithi183 ceph-mon[34589]: pgmap v378: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 39 op/s 2024-09-05T01:16:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:16 smithi129 ceph-mon[30979]: pgmap v379: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.8 MiB/s wr, 30 op/s 2024-09-05T01:16:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:16 smithi194 ceph-mon[34941]: pgmap v379: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.8 MiB/s wr, 30 op/s 2024-09-05T01:16:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:16 smithi183 ceph-mon[34589]: pgmap v379: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.8 MiB/s wr, 30 op/s 2024-09-05T01:16:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 -- 172.21.15.129:0/3891884239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff990109a10 msgr2=0x7ff99010bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 --2- 172.21.15.129:0/3891884239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff990109a10 0x7ff99010bea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff988009ca0 tx=0x7ff98802a950 comp rx=0 tx=0).stop 2024-09-05T01:16:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 -- 172.21.15.129:0/3891884239 shutdown_connections 2024-09-05T01:16:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 --2- 172.21.15.129:0/3891884239 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff990109a10 0x7ff99010bea0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 --2- 172.21.15.129:0/3891884239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 --2- 172.21.15.129:0/3891884239 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff9900761b0 0x7ff990074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.229+0000 7ff997653640 1 -- 172.21.15.129:0/3891884239 >> 172.21.15.129:0/3891884239 conn(0x7ff9900fd2e0 msgr2=0x7ff9900ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- 172.21.15.129:0/3891884239 shutdown_connections 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- 172.21.15.129:0/3891884239 wait complete. 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 Processor -- start 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- start start 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff990109a10 0x7ff9901a1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010f890 con 0x7ff9900761b0 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010f710 con 0x7ff990074bd0 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff997653640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff99010fa10 con 0x7ff990109a10 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff994bee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff994bee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60148/0 (socket says 172.21.15.129:60148) 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.230+0000 7ff994bee640 1 -- 172.21.15.129:0/26457348 learned_addr learned my addr 172.21.15.129:0/26457348 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff995bf0640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff990109a10 0x7ff9901a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff9953ef640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 -- 172.21.15.129:0/26457348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff990109a10 msgr2=0x7ff9901a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff990109a10 0x7ff9901a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 -- 172.21.15.129:0/26457348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 msgr2=0x7ff990199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 -- 172.21.15.129:0/26457348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9901a1ae0 con 0x7ff9900761b0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff9953ef640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff994bee640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d950 tx=0x7ff98c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff9867fc640 1 -- 172.21.15.129:0/26457348 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c017070 con 0x7ff9900761b0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff9867fc640 1 -- 172.21.15.129:0/26457348 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98c009c30 con 0x7ff9900761b0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9901a1dd0 con 0x7ff9900761b0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff9867fc640 1 -- 172.21.15.129:0/26457348 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00f620 con 0x7ff9900761b0 2024-09-05T01:16:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.231+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9901a2290 con 0x7ff9900761b0 2024-09-05T01:16:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.234+0000 7ff9867fc640 1 -- 172.21.15.129:0/26457348 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7ff98c0109a0 con 0x7ff9900761b0 2024-09-05T01:16:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.234+0000 7ff99a3e9740 1 Processor -- start 2024-09-05T01:16:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 -- start start 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec4cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec4f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56506ebab2f0 con 0x56506ec479a0 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56506eb53270 con 0x56506ec482e0 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56506eba72b0 con 0x56506ec4d370 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60160/0 (socket says 172.21.15.129:60160) 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 -- 172.21.15.129:0/4284981543 learned_addr learned my addr 172.21.15.129:0/4284981543 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff994bee640 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec4f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff9953ef640 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec4cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 -- 172.21.15.129:0/4284981543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 msgr2=0x56506ec4f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec4f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 -- 172.21.15.129:0/4284981543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 msgr2=0x56506ec4cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec4cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff995bf0640 1 -- 172.21.15.129:0/4284981543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506eb52340 con 0x56506ec479a0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.235+0000 7ff9867fc640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 0x7ff97c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff9867fc640 1 -- 172.21.15.129:0/26457348 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7ff98c0cf2c0 con 0x7ff9900761b0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff995bf0640 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff988005e30 tx=0x7ff988009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69ca22d5047a6b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff995bf0640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 0x7ff97c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff97b7fe640 1 -- 172.21.15.129:0/4284981543 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988049040 con 0x56506ec479a0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff97b7fe640 1 -- 172.21.15.129:0/4284981543 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98802f480 con 0x56506ec479a0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.236+0000 7ff97b7fe640 1 -- 172.21.15.129:0/4284981543 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988034040 con 0x56506ec479a0 2024-09-05T01:16:18.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- 172.21.15.129:0/4284981543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 msgr2=0x56506ec47da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff988005e30 tx=0x7ff988009a20 comp rx=0 tx=0).stop 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff995bf0640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 0x7ff97c0a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff988044000 tx=0x7ff988044780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- 172.21.15.129:0/4284981543 shutdown_connections 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec4f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec4cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/4284981543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec47da0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- 172.21.15.129:0/4284981543 >> 172.21.15.129:0/4284981543 conn(0x56506ec43010 msgr2=0x56506ec43410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- 172.21.15.129:0/4284981543 shutdown_connections 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- 172.21.15.129:0/4284981543 wait complete. 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 -- start start 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.237+0000 7ff99a3e9740 1 Processor -- start 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 0x56506ec520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec4d370 0x56506ec59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56506ec502e0 con 0x56506ec479a0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56506eb34290 con 0x56506ec4d370 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56506ec50460 con 0x56506ec482e0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 10 client.0 _ll_register_callbacks cb 0x56506eb65140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60176/0 (socket says 172.21.15.129:60176) 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 -- 172.21.15.129:0/637531070 learned_addr learned my addr 172.21.15.129:0/637531070 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff995bf0640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec4d370 0x56506ec59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff994bee640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 0x56506ec520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 -- 172.21.15.129:0/637531070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 msgr2=0x56506ec520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 0x56506ec520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 -- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec4d370 msgr2=0x56506ec59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec4d370 0x56506ec59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 -- 172.21.15.129:0/637531070 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506e8be210 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff994bee640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 0x56506ec520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff9953ef640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff980010670 tx=0x7ff980010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451abcffa872add6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98000ead0 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7ff98000ec70 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98802f480 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.238+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff980025050 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56506e98a620 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98802f480 con 0x56506ec479a0 2024-09-05T01:16:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7ff98000f040 con 0x56506ec479a0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 20 client.15444 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ec5d8b0 0x56506ec5fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 4 client.15444 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56506ebb8a20 con 0x56506ec5d8b0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.239+0000 7ff994bee640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ec5d8b0 0x56506ec5fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.240+0000 7ff994bee640 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ec5d8b0 0x56506ec5fd20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc294bb585b197c1 server_cookie=2fc22750283e0169 in_seq=0 out_seq=0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.240+0000 7ff978ff9640 10 client.15444 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.240+0000 7ff978ff9640 1 -- 172.21.15.129:0/637531070 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56506ebb8a20 con 0x56506ec5d8b0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.240+0000 7ff978ff9640 10 client.15444 handle_command_reply: tid=0 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.240+0000 7ff99a3e9740 1 client.15444 shutdown 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 msgr2=0x56506ec51b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff980010670 tx=0x7ff980010b30 comp rx=0 tx=0).stop 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 shutdown_connections 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ec5d8b0 0x56506ec5fd20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec4d370 0x56506ec59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff978ff9640 0 client.15444 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec482e0 0x56506ec520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/637531070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec51b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 >> 172.21.15.129:0/637531070 conn(0x56506ec43010 msgr2=0x56506ec44820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 shutdown_connections 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 1 -- 172.21.15.129:0/637531070 wait complete. 2024-09-05T01:16:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.241+0000 7ff99a3e9740 20 client.15444 trim_cache size 0 max 16384 2024-09-05T01:16:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 Processor -- start 2024-09-05T01:16:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 -- start start 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 0x56506ec52160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901af380 con 0x56506ec4d370 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901acc50 con 0x56506ec482e0 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901aaaa0 con 0x56506ec479a0 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff995bf0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff995bf0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60186/0 (socket says 172.21.15.129:60186) 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.255+0000 7ff995bf0640 1 -- 172.21.15.129:0/478060892 learned_addr learned my addr 172.21.15.129:0/478060892 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff9953ef640 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 0x56506ec52160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 -- 172.21.15.129:0/478060892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 msgr2=0x56506ec52160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 0x56506ec52160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 -- 172.21.15.129:0/478060892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 msgr2=0x56506ec590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 -- 172.21.15.129:0/478060892 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56506ec487c0 con 0x56506ec4d370 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff9953ef640 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 0x56506ec52160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff995bf0640 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff988030040 tx=0x7ff988050000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2095f5eaa830ba02 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff97affd640 1 -- 172.21.15.129:0/478060892 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988056040 con 0x56506ec4d370 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff97affd640 1 -- 172.21.15.129:0/478060892 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98805f070 con 0x56506ec4d370 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.256+0000 7ff97affd640 1 -- 172.21.15.129:0/478060892 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988034040 con 0x56506ec4d370 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- 172.21.15.129:0/478060892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 msgr2=0x56506ec420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff988030040 tx=0x7ff988050000 comp rx=0 tx=0).stop 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- 172.21.15.129:0/478060892 shutdown_connections 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec4d370 0x56506ec420b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/478060892 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec479a0 0x56506ec52160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- 172.21.15.129:0/478060892 >> 172.21.15.129:0/478060892 conn(0x56506ec43010 msgr2=0x56506ec419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- 172.21.15.129:0/478060892 shutdown_connections 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- 172.21.15.129:0/478060892 wait complete. 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- start start 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 Processor -- start 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec58f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec57040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ab110 con 0x56506ec479a0 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ac580 con 0x56506ec482e0 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff9901ab7e0 con 0x56506ec4d370 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff99a3e9740 10 client.0 _ll_register_callbacks cb 0x7ff9800094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff9953ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff9953ef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60192/0 (socket says 172.21.15.129:60192) 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.257+0000 7ff9953ef640 1 -- 172.21.15.129:0/132476167 learned_addr learned my addr 172.21.15.129:0/132476167 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff994bee640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec58f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 -- 172.21.15.129:0/132476167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 msgr2=0x56506ec57040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff995bf0640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec57040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec57040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 -- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 msgr2=0x56506ec58f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec58f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 -- 172.21.15.129:0/132476167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56506ec576e0 con 0x56506ec479a0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff994bee640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff995bf0640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec57040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff9953ef640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff980012f00 tx=0x7ff980009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836db331206a3185 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff978ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98002d030 con 0x56506ec479a0 2024-09-05T01:16:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7ff980022040 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98805f070 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff980010e70 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff99a3e9740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56506ecc8f90 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.258+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff98805f070 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7ff980004650 con 0x56506ec479a0 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 20 client.15450 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ecc9970 0x56506eccbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 4 client.15450 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56506ebb2d70 con 0x56506ecc9970 2024-09-05T01:16:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff994bee640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ecc9970 0x56506eccbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff994bee640 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ecc9970 0x56506eccbd90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf18a5d85a008b4a server_cookie=ccfae531a3d1f78e in_seq=0 out_seq=0 2024-09-05T01:16:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.259+0000 7ff978ff9640 10 client.15450 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.272+0000 7ff978ff9640 1 -- 172.21.15.129:0/132476167 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56506ebb2d70 con 0x56506ecc9970 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.272+0000 7ff978ff9640 10 client.15450 handle_command_reply: tid=0 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 client.15450 shutdown 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 msgr2=0x56506ec589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff980012f00 tx=0x7ff980009e20 comp rx=0 tx=0).stop 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 shutdown_connections 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56506ecc9970 0x56506eccbd90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56506ec4d370 0x56506ec57040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56506ec482e0 0x56506ec58f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 --2- 172.21.15.129:0/132476167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56506ec479a0 0x56506ec589e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff978ff9640 0 client.15450 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 >> 172.21.15.129:0/132476167 conn(0x56506ec43010 msgr2=0x56506ec421c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 shutdown_connections 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 1 -- 172.21.15.129:0/132476167 wait complete. 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff99a3e9740 20 client.15450 trim_cache size 0 max 16384 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:16:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:16:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 msgr2=0x7ff97c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.273+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 0x7ff97c0a5190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff988044000 tx=0x7ff988044780 comp rx=0 tx=0).stop 2024-09-05T01:16:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 msgr2=0x7ff99019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d950 tx=0x7ff98c00de10 comp rx=0 tx=0).stop 2024-09-05T01:16:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 shutdown_connections 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff97c0a2cd0 0x7ff97c0a5190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff990109a10 0x7ff9901a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9900761b0 0x7ff99019a400 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 --2- 172.21.15.129:0/26457348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff990074bd0 0x7ff990199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 >> 172.21.15.129:0/26457348 conn(0x7ff9900fd2e0 msgr2=0x7ff9901059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 shutdown_connections 2024-09-05T01:16:18.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:18.274+0000 7ff997653640 1 -- 172.21.15.129:0/26457348 wait complete. 2024-09-05T01:16:19.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:18 smithi129 ceph-mon[30979]: pgmap v380: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.9 MiB/s wr, 28 op/s 2024-09-05T01:16:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:18 smithi194 ceph-mon[34941]: pgmap v380: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.9 MiB/s wr, 28 op/s 2024-09-05T01:16:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:18 smithi183 ceph-mon[34589]: pgmap v380: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 2.9 MiB/s wr, 28 op/s 2024-09-05T01:16:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:20 smithi129 ceph-mon[30979]: pgmap v381: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-09-05T01:16:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:20 smithi194 ceph-mon[34941]: pgmap v381: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-09-05T01:16:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:20 smithi183 ceph-mon[34589]: pgmap v381: 129 pgs: 129 active+clean; 227 MiB data, 898 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 MiB/s wr, 19 op/s 2024-09-05T01:16:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:22 smithi129 ceph-mon[30979]: pgmap v382: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 432 KiB/s wr, 8 op/s 2024-09-05T01:16:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:22 smithi194 ceph-mon[34941]: pgmap v382: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 432 KiB/s wr, 8 op/s 2024-09-05T01:16:23.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:22 smithi183 ceph-mon[34589]: pgmap v382: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 432 KiB/s wr, 8 op/s 2024-09-05T01:16:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:24 smithi129 ceph-mon[30979]: pgmap v383: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 401 KiB/s wr, 11 op/s 2024-09-05T01:16:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:24 smithi194 ceph-mon[34941]: pgmap v383: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 401 KiB/s wr, 11 op/s 2024-09-05T01:16:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:24 smithi183 ceph-mon[34589]: pgmap v383: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 401 KiB/s wr, 11 op/s 2024-09-05T01:16:26.744 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:16:27.008 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:16:27.129 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:26 smithi129 ceph-mon[30979]: pgmap v384: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-05T01:16:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:26 smithi194 ceph-mon[34941]: pgmap v384: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-05T01:16:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:26 smithi183 ceph-mon[34589]: pgmap v384: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-09-05T01:16:27.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.508+0000 7fd4aa150640 1 -- 172.21.15.129:0/302185060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4102790 msgr2=0x7fd4a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:27.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.508+0000 7fd4aa150640 1 --2- 172.21.15.129:0/302185060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4102790 0x7fd4a41067e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd498009900 tx=0x7fd49802a410 comp rx=0 tx=0).stop 2024-09-05T01:16:27.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.508+0000 7fd4aa150640 1 -- 172.21.15.129:0/302185060 shutdown_connections 2024-09-05T01:16:27.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.508+0000 7fd4aa150640 1 --2- 172.21.15.129:0/302185060 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4106f10 0x7fd4a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.508+0000 7fd4aa150640 1 --2- 172.21.15.129:0/302185060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4102790 0x7fd4a41067e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 --2- 172.21.15.129:0/302185060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4101de0 0x7fd4a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 -- 172.21.15.129:0/302185060 >> 172.21.15.129:0/302185060 conn(0x7fd4a40fd8c0 msgr2=0x7fd4a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 -- 172.21.15.129:0/302185060 shutdown_connections 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 -- 172.21.15.129:0/302185060 wait complete. 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 Processor -- start 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.509+0000 7fd4aa150640 1 -- start start 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4102790 0x7fd4a4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 0x7fd4a419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111de0 con 0x7fd4a4101de0 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111c60 con 0x7fd4a4106f10 2024-09-05T01:16:27.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a4111f60 con 0x7fd4a4102790 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38850/0 (socket says 172.21.15.129:38850) 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 -- 172.21.15.129:0/241838454 learned_addr learned my addr 172.21.15.129:0/241838454 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a2ffd640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4102790 0x7fd4a4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a3fff640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 0x7fd4a419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 -- 172.21.15.129:0/241838454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4102790 msgr2=0x7fd4a4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4102790 0x7fd4a4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 -- 172.21.15.129:0/241838454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 msgr2=0x7fd4a419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 0x7fd4a419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 -- 172.21.15.129:0/241838454 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a419d290 con 0x7fd4a4101de0 2024-09-05T01:16:27.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a3fff640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 0x7fd4a419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a37fe640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd49400cce0 tx=0x7fd49400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400f650 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a419d580 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd49400fca0 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494018b90 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.510+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a41a4e60 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.512+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a41021c0 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.512+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7fd49400f7f0 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.513+0000 7fd4a0ff9640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 0x7fd48c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.513+0000 7fd4a2ffd640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 0x7fd48c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.513+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fd4940cfad0 con 0x7fd4a4101de0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.514+0000 7fd4a2ffd640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 0x7fd48c0a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd498031440 tx=0x7fd498005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:27.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.517+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd494014030 con 0x7fd4a4101de0 2024-09-05T01:16:27.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.716+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4a410a0f0 con 0x7fd4a4101de0 2024-09-05T01:16:27.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.717+0000 7fd4a0ff9640 1 -- 172.21.15.129:0/241838454 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7fd49409c4c0 con 0x7fd4a4101de0 2024-09-05T01:16:27.718 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:16:27.718 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"btime":"2024-09-05T01:15:41:859752+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:41.039933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:16:27.718 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 27 2024-09-05T01:16:27.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 msgr2=0x7fd48c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:27.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 0x7fd48c0a50d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd498031440 tx=0x7fd498005b50 comp rx=0 tx=0).stop 2024-09-05T01:16:27.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 msgr2=0x7fd4a41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:27.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd49400cce0 tx=0x7fd49400a620 comp rx=0 tx=0).stop 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 shutdown_connections 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd48c0a2c10 0x7fd48c0a50d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd4a4106f10 0x7fd4a419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd4a4102790 0x7fd4a4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 --2- 172.21.15.129:0/241838454 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd4a4101de0 0x7fd4a41982e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 >> 172.21.15.129:0/241838454 conn(0x7fd4a40fd8c0 msgr2=0x7fd4a40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 shutdown_connections 2024-09-05T01:16:27.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:27.720+0000 7fd4aa150640 1 -- 172.21.15.129:0/241838454 wait complete. 2024-09-05T01:16:27.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:27 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/241838454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:27 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/241838454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:28.312 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:27 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/241838454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:16:28.480 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 83ea5152 --release --await' 2024-09-05T01:16:28.481 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 83ea5152 --release --await 2024-09-05T01:16:28.727 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:16:29.131 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:28 smithi129 ceph-mon[30979]: pgmap v385: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:29.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 -- 172.21.15.129:0/2467854598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50100a60 msgr2=0x7f4c50100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 --2- 172.21.15.129:0/2467854598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50100a60 0x7f4c50100ec0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009930 tx=0x7f4c3802a420 comp rx=0 tx=0).stop 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 -- 172.21.15.129:0/2467854598 shutdown_connections 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 --2- 172.21.15.129:0/2467854598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50101400 0x7f4c5010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 --2- 172.21.15.129:0/2467854598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50100a60 0x7f4c50100ec0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 --2- 172.21.15.129:0/2467854598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50106a60 0x7f4c50106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 -- 172.21.15.129:0/2467854598 >> 172.21.15.129:0/2467854598 conn(0x7f4c500fc780 msgr2=0x7f4c500feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 -- 172.21.15.129:0/2467854598 shutdown_connections 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.216+0000 7f4c55479640 1 -- 172.21.15.129:0/2467854598 wait complete. 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 Processor -- start 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 -- start start 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50100a60 0x7f4c50197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50101400 0x7f4c50197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50110db0 con 0x7f4c50106a60 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50110c30 con 0x7f4c50101400 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c55479640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c50110f30 con 0x7f4c50100a60 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38868/0 (socket says 172.21.15.129:38868) 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/206654463 learned_addr learned my addr 172.21.15.129:0/206654463 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.217+0000 7f4c4effd640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50100a60 0x7f4c50197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c467fc640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50101400 0x7f4c50197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/206654463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50100a60 msgr2=0x7f4c50197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50100a60 0x7f4c50197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/206654463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50101400 msgr2=0x7f4c50197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50101400 0x7f4c50197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/206654463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5019c3c0 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b970 tx=0x7f4c4800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4cff9640 1 -- 172.21.15.129:0/206654463 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c48019070 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5019c6b0 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4cff9640 1 -- 172.21.15.129:0/206654463 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c48014440 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c4cff9640 1 -- 172.21.15.129:0/206654463 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480075d0 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.218+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c501a3f90 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.220+0000 7f4c4cff9640 1 -- 172.21.15.129:0/206654463 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f4c4801c070 con 0x7f4c50106a60 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.220+0000 7f4c57852740 1 Processor -- start 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 -- start start 2024-09-05T01:16:29.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4cff9640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 0x7f4c3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf28638c0 0x556bf2862150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2862af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556bf27c5620 con 0x556bf286f300 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556bf276ce50 con 0x556bf28638c0 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c57852740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bf27c15e0 con 0x556bf2862690 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4cff9640 1 -- 172.21.15.129:0/206654463 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f4c480ceb40 con 0x7f4c50106a60 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c467fc640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 0x7f4c3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38878/0 (socket says 172.21.15.129:38878) 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/3099173161 learned_addr learned my addr 172.21.15.129:0/3099173161 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:29.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2862af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.221+0000 7f4c4effd640 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf28638c0 0x556bf2862150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/3099173161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 msgr2=0x556bf2862af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2862af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/3099173161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf28638c0 msgr2=0x556bf2862150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf28638c0 0x556bf2862150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/3099173161 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf275dd40 con 0x556bf286f300 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c467fc640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 0x7f4c3c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c38000c00 tx=0x7f4c38005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c48097910 tx=0x7f4c48004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcbf83cb0ff7880 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c457fa640 1 -- 172.21.15.129:0/3099173161 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4809b640 con 0x556bf286f300 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c457fa640 1 -- 172.21.15.129:0/3099173161 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4809af40 con 0x556bf286f300 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.222+0000 7f4c457fa640 1 -- 172.21.15.129:0/3099173161 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480cc330 con 0x556bf286f300 2024-09-05T01:16:29.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- 172.21.15.129:0/3099173161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 msgr2=0x556bf28716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4c48097910 tx=0x7f4c48004b50 comp rx=0 tx=0).stop 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- 172.21.15.129:0/3099173161 shutdown_connections 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf28716f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2862af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- 172.21.15.129:0/3099173161 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf28638c0 0x556bf2862150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- 172.21.15.129:0/3099173161 >> 172.21.15.129:0/3099173161 conn(0x556bf285e800 msgr2=0x556bf285ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- 172.21.15.129:0/3099173161 shutdown_connections 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- 172.21.15.129:0/3099173161 wait complete. 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- start start 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 Processor -- start 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf286c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf28638c0 0x556bf286c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556bf2872150 con 0x556bf286f300 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556bf274df10 con 0x556bf2862690 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556bf28722d0 con 0x556bf28638c0 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.223+0000 7f4c57852740 10 client.0 _ll_register_callbacks cb 0x556bf2771db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38882/0 (socket says 172.21.15.129:38882) 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/2116929350 learned_addr learned my addr 172.21.15.129:0/2116929350 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4effd640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf286c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c467fc640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf28638c0 0x556bf286c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/2116929350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf28638c0 msgr2=0x556bf286c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf28638c0 0x556bf286c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 msgr2=0x556bf286c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf286c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 -- 172.21.15.129:0/2116929350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf24d4580 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4c480a3680 tx=0x7f4c480cc560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec1e7b97da23f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c480a3920 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f4c480b9b80 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c4809af40 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c48084270 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c57852740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bf259e960 con 0x556bf286f300 2024-09-05T01:16:29.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c4809af40 con 0x556bf286f300 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.224+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f4c4809a390 con 0x556bf286f300 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 10 client.15468 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 20 client.15468 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf287f080 0x556bf2881470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 4 client.15468 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bf27d2b90 con 0x556bf287f080 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c4effd640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf287f080 0x556bf2881470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c367fc640 10 client.15468 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.225+0000 7f4c4effd640 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf287f080 0x556bf2881470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed904f020428a265 server_cookie=f784b89576d5abbe in_seq=0 out_seq=0 2024-09-05T01:16:29.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c367fc640 1 -- 172.21.15.129:0/2116929350 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bf27d2b90 con 0x556bf287f080 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c367fc640 10 client.15468 handle_command_reply: tid=0 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 client.15468 shutdown 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 msgr2=0x556bf286cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4c480a3680 tx=0x7f4c480cc560 comp rx=0 tx=0).stop 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 shutdown_connections 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf287f080 0x556bf2881470 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf286f300 0x556bf286cd80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf28638c0 0x556bf286c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 --2- 172.21.15.129:0/2116929350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf286c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c367fc640 0 client.15468 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 >> 172.21.15.129:0/2116929350 conn(0x556bf285e800 msgr2=0x556bf2864560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 shutdown_connections 2024-09-05T01:16:29.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 1 -- 172.21.15.129:0/2116929350 wait complete. 2024-09-05T01:16:29.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.226+0000 7f4c57852740 20 client.15468 trim_cache size 0 max 16384 2024-09-05T01:16:29.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 Processor -- start 2024-09-05T01:16:29.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 -- start start 2024-09-05T01:16:29.244 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf285d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 0x556bf2886fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501af5d0 con 0x556bf28638c0 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501acea0 con 0x556bf2862690 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c57852740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501aacf0 con 0x556bf286f300 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c467fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c467fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38892/0 (socket says 172.21.15.129:38892) 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.243+0000 7f4c467fc640 1 -- 172.21.15.129:0/3005694675 learned_addr learned my addr 172.21.15.129:0/3005694675 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 0x556bf2886fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 -- 172.21.15.129:0/3005694675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 msgr2=0x556bf2886fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 0x556bf2886fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 -- 172.21.15.129:0/3005694675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 msgr2=0x556bf285d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf285d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 -- 172.21.15.129:0/3005694675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bf285abb0 con 0x556bf28638c0 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 0x556bf2886fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009930 tx=0x7f4c38046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b477577e3c47cc5f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c44ff9640 1 -- 172.21.15.129:0/3005694675 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38035050 con 0x556bf28638c0 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c44ff9640 1 -- 172.21.15.129:0/3005694675 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c38004490 con 0x556bf28638c0 2024-09-05T01:16:29.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.244+0000 7f4c44ff9640 1 -- 172.21.15.129:0/3005694675 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38049b10 con 0x556bf28638c0 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- 172.21.15.129:0/3005694675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 msgr2=0x556bf2886a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009930 tx=0x7f4c38046000 comp rx=0 tx=0).stop 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- 172.21.15.129:0/3005694675 shutdown_connections 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf286f300 0x556bf2886fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2886a60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- 172.21.15.129:0/3005694675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf2862690 0x556bf285d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- 172.21.15.129:0/3005694675 >> 172.21.15.129:0/3005694675 conn(0x556bf285e800 msgr2=0x556bf285d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- 172.21.15.129:0/3005694675 shutdown_connections 2024-09-05T01:16:29.249 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- 172.21.15.129:0/3005694675 wait complete. 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- start start 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 Processor -- start 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2877b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf286f300 0x556bf2874a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501ab360 con 0x556bf28638c0 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501ac7d0 con 0x556bf286f300 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.245+0000 7f4c57852740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c501aba30 con 0x556bf2862690 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c57852740 10 client.0 _ll_register_callbacks cb 0x7f4c48097290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38900/0 (socket says 172.21.15.129:38900) 2024-09-05T01:16:29.250 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 -- 172.21.15.129:0/3746921257 learned_addr learned my addr 172.21.15.129:0/3746921257 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c4effd640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2877b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c4f7fe640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf286f300 0x556bf2874a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 -- 172.21.15.129:0/3746921257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 msgr2=0x556bf2877b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2877b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 -- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf286f300 msgr2=0x556bf2874a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf286f300 0x556bf2874a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 -- 172.21.15.129:0/3746921257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bf28750b0 con 0x556bf28638c0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c467fc640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006010 tx=0x7f4c380058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19bc045fa696bfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c367fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3805a070 con 0x556bf28638c0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f4c380035f0 con 0x556bf28638c0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.246+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c38004490 con 0x556bf28638c0 2024-09-05T01:16:29.251 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4c38066050 con 0x556bf28638c0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 10 client.15474 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bf28ea820 con 0x556bf28638c0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c38004490 con 0x556bf28638c0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f4c3803b630 con 0x556bf28638c0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 10 client.15474 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 10 client.15474 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 20 client.15474 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf28eaec0 0x556bf28ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 4 client.15474 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "83ea5152", "release": true, "await": true}] 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "83ea5152", "release": true, "await": true}) -- 0x556bf27cd0a0 con 0x556bf28eaec0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.247+0000 7f4c4effd640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf28eaec0 0x556bf28ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:29.252 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.248+0000 7f4c367fc640 10 client.15474 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:29.253 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.248+0000 7f4c4effd640 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf28eaec0 0x556bf28ed2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70fe75995fbd4705 server_cookie=d6e6316c69d83bc9 in_seq=0 out_seq=0 2024-09-05T01:16:29.253 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:28 smithi194 ceph-mon[34941]: pgmap v385: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:29.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.301+0000 7f4c367fc640 1 -- 172.21.15.129:0/3746921257 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556bf27cd0a0 con 0x556bf28eaec0 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.301+0000 7f4c367fc640 10 client.15474 handle_command_reply: tid=0 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.301+0000 7f4c57852740 1 client.15474 shutdown 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.301+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 msgr2=0x556bf2874530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.301+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c38006010 tx=0x7f4c380058d0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 shutdown_connections 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556bf28eaec0 0x556bf28ed2e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556bf286f300 0x556bf2874a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556bf28638c0 0x556bf2874530 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 --2- 172.21.15.129:0/3746921257 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556bf2862690 0x556bf2877b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c367fc640 0 client.15474 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:29.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 >> 172.21.15.129:0/3746921257 conn(0x556bf285e800 msgr2=0x556bf286db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 shutdown_connections 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 1 -- 172.21.15.129:0/3746921257 wait complete. 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c57852740 20 client.15474 trim_cache size 0 max 16384 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 27, 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 4, 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:16:29.304 INFO:teuthology.orchestra.run.smithi129.stdout: "83ea5152": { 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 4, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 148.2, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:16:29.305 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:16:29.306 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 msgr2=0x7f4c3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 0x7f4c3c0a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4c38000c00 tx=0x7f4c38005d50 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 msgr2=0x7f4c5019bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.302+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4c4800b970 tx=0x7f4c4800be30 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 shutdown_connections 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c3c0a2c10 0x7f4c3c0a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c50106a60 0x7f4c5019bce0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c50101400 0x7f4c50197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 --2- 172.21.15.129:0/206654463 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c50100a60 0x7f4c50197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 >> 172.21.15.129:0/206654463 conn(0x7f4c500fc780 msgr2=0x7f4c500fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 shutdown_connections 2024-09-05T01:16:29.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:16:29.303+0000 7f4c55479640 1 -- 172.21.15.129:0/206654463 wait complete. 2024-09-05T01:16:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:28 smithi183 ceph-mon[34589]: pgmap v385: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:29.745 INFO:teuthology.orchestra.run.smithi129.stderr:900000 of 3200000 tuples (28%) done (elapsed 34.21 s, remaining 87.41 s) 2024-09-05T01:16:30.071 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '83ea5152', total seconds elapsed: 31.6 2024-09-05T01:16:30.789 INFO:teuthology.orchestra.run.smithi129.stderr:1000000 of 3200000 tuples (31%) done (elapsed 35.25 s, remaining 77.55 s) 2024-09-05T01:16:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:30 smithi129 ceph-mon[30979]: pgmap v386: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:30 smithi194 ceph-mon[34941]: pgmap v386: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.378+0000 7f77aab18640 1 -- 172.21.15.129:0/1831580760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a4101c30 msgr2=0x7f77a41020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.378+0000 7f77aab18640 1 --2- 172.21.15.129:0/1831580760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a4101c30 0x7f77a41020b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7790009900 tx=0x7f779002a410 comp rx=0 tx=0).stop 2024-09-05T01:16:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.378+0000 7f77aab18640 1 -- 172.21.15.129:0/1831580760 shutdown_connections 2024-09-05T01:16:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- 172.21.15.129:0/1831580760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a41025f0 0x7f77a4108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- 172.21.15.129:0/1831580760 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a4101c30 0x7f77a41020b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- 172.21.15.129:0/1831580760 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4100a40 0x7f77a4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- 172.21.15.129:0/1831580760 >> 172.21.15.129:0/1831580760 conn(0x7f77a40fc1e0 msgr2=0x7f77a40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- 172.21.15.129:0/1831580760 shutdown_connections 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- 172.21.15.129:0/1831580760 wait complete. 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 Processor -- start 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- start start 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 0x7f77a4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101c30 0x7f77a4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410c570 con 0x7f77a41025f0 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410c3f0 con 0x7f77a4100a40 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.379+0000 7f77aab18640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a410c6f0 con 0x7f77a4101c30 2024-09-05T01:16:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38930/0 (socket says 172.21.15.129:38930) 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a90b5640 1 -- 172.21.15.129:0/2188732227 learned_addr learned my addr 172.21.15.129:0/2188732227 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 0x7f77a4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101c30 0x7f77a4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 -- 172.21.15.129:0/2188732227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101c30 msgr2=0x7f77a4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101c30 0x7f77a4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 -- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 msgr2=0x7f77a419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 -- 172.21.15.129:0/2188732227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a419f960 con 0x7f77a4100a40 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 0x7f77a4197d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f779800e9c0 tx=0x7f779800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a1ffb640 1 -- 172.21.15.129:0/2188732227 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800ccc0 con 0x7f77a4100a40 2024-09-05T01:16:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a419fc50 con 0x7f77a4100a40 2024-09-05T01:16:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.380+0000 7f77a1ffb640 1 -- 172.21.15.129:0/2188732227 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77980035f0 con 0x7f77a4100a40 2024-09-05T01:16:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.381+0000 7f77a1ffb640 1 -- 172.21.15.129:0/2188732227 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800ce60 con 0x7f77a4100a40 2024-09-05T01:16:31.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.381+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a41a01a0 con 0x7f77a4100a40 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77a1ffb640 1 -- 172.21.15.129:0/2188732227 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7f77980037a0 con 0x7f77a4100a40 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 Processor -- start 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 -- start start 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf4153da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaf40b22f0 con 0x55aaf414f2e0 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaf405a270 con 0x55aaf414e9a0 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.383+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aaf40ae2b0 con 0x55aaf4154370 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a88b4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf4153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:49564/0 (socket says 172.21.15.129:49564) 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:34196/0 (socket says 172.21.15.129:34196) 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 -- 172.21.15.129:0/1218383735 learned_addr learned my addr 172.21.15.129:0/1218383735 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a1ffb640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 0x7f77940a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 -- 172.21.15.129:0/1218383735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 msgr2=0x55aaf4156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 -- 172.21.15.129:0/1218383735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 msgr2=0x55aaf4153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf4153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 -- 172.21.15.129:0/1218383735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf4059340 con 0x55aaf414e9a0 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a3fff640 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a1ffb640 1 -- 172.21.15.129:0/2188732227 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f77980cfa40 con 0x7f77a4100a40 2024-09-05T01:16:31.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 0x7f77940a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779c011a30 tx=0x7f779c011ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a8cf3fbcd9ae4eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f778effd640 1 -- 172.21.15.129:0/1218383735 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x55aaf414e9a0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 0x7f77940a5190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f779c023000 tx=0x7f779c024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f778effd640 1 -- 172.21.15.129:0/1218383735 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779c01cc00 con 0x55aaf414e9a0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.384+0000 7f778effd640 1 -- 172.21.15.129:0/1218383735 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c00e3b0 con 0x55aaf414e9a0 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 -- 172.21.15.129:0/1218383735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 msgr2=0x55aaf414eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f779c011a30 tx=0x7f779c011ef0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 -- 172.21.15.129:0/1218383735 shutdown_connections 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf4154370 0x55aaf4156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf4153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/1218383735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf414eda0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 -- 172.21.15.129:0/1218383735 >> 172.21.15.129:0/1218383735 conn(0x55aaf414a010 msgr2=0x55aaf414a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 -- 172.21.15.129:0/1218383735 shutdown_connections 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.385+0000 7f77ad8b2740 1 -- 172.21.15.129:0/1218383735 wait complete. 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 -- start start 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 Processor -- start 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf4159070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aaf4157340 con 0x55aaf4154370 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55aaf403b290 con 0x55aaf414e9a0 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55aaf41574c0 con 0x55aaf414f2e0 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77ad8b2740 10 client.0 _ll_register_callbacks cb 0x55aaf406c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a88b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38944/0 (socket says 172.21.15.129:38944) 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a88b4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:49578/0 (socket says 172.21.15.129:49578) 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a90b5640 1 -- 172.21.15.129:0/232424417 learned_addr learned my addr 172.21.15.129:0/232424417 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.386+0000 7f77a3fff640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 -- 172.21.15.129:0/232424417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 msgr2=0x55aaf4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 -- 172.21.15.129:0/232424417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 msgr2=0x55aaf41600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 -- 172.21.15.129:0/232424417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf3dc5210 con 0x55aaf414e9a0 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77a88b4640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77980ccf80 tx=0x7f77980cd4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4e9d2929297f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:16:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798099e10 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f77980a46c0 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779c01cc00 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7798003b10 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77ad8b2740 10 client.25049 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf3e91620 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779c01cc00 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.387+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f7798004190 con 0x55aaf414e9a0 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 10 client.25049 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 10 client.25049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 20 client.25049 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41648a0 0x55aaf4166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 4 client.25049 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaf40bfa20 con 0x55aaf41648a0 2024-09-05T01:16:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77a3fff640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41648a0 0x55aaf4166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f77a3fff640 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41648a0 0x55aaf4166c90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c02bb86bec05a72e server_cookie=bd748098e17e46c0 in_seq=0 out_seq=0 2024-09-05T01:16:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.388+0000 7f7783fff640 10 client.25049 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f7783fff640 1 -- 172.21.15.129:0/232424417 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aaf40bfa20 con 0x55aaf41648a0 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f7783fff640 10 client.25049 handle_command_reply: tid=0 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 client.25049 shutdown 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 msgr2=0x55aaf4158b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f77980ccf80 tx=0x7f77980cd4c0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 shutdown_connections 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41648a0 0x55aaf4166c90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f7783fff640 0 client.25049 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/232424417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf4158b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 >> 172.21.15.129:0/232424417 conn(0x55aaf414a010 msgr2=0x55aaf414b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 shutdown_connections 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 1 -- 172.21.15.129:0/232424417 wait complete. 2024-09-05T01:16:31.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.389+0000 7f77ad8b2740 20 client.25049 trim_cache size 0 max 16384 2024-09-05T01:16:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 Processor -- start 2024-09-05T01:16:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 -- start start 2024-09-05T01:16:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 0x55aaf41490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf4154370 0x55aaf416ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41acfb0 con 0x55aaf414f2e0 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41aa880 con 0x55aaf4154370 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41a86d0 con 0x55aaf414e9a0 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38946/0 (socket says 172.21.15.129:38946) 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 -- 172.21.15.129:0/2012674371 learned_addr learned my addr 172.21.15.129:0/2012674371 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 0x55aaf41490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf4154370 0x55aaf416ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 -- 172.21.15.129:0/2012674371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 msgr2=0x55aaf41490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 0x55aaf41490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 -- 172.21.15.129:0/2012674371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf4154370 msgr2=0x55aaf416ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf4154370 0x55aaf416ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a3fff640 1 -- 172.21.15.129:0/2012674371 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaf4167000 con 0x55aaf414f2e0 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.405+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 0x55aaf41490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f779002a930 tx=0x7f7790009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7eebfb93722d684 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f778e7fc640 1 -- 172.21.15.129:0/2012674371 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790035040 con 0x55aaf414f2e0 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f778e7fc640 1 -- 172.21.15.129:0/2012674371 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7790004240 con 0x55aaf414f2e0 2024-09-05T01:16:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f778e7fc640 1 -- 172.21.15.129:0/2012674371 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790046620 con 0x55aaf414f2e0 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2012674371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 msgr2=0x55aaf416b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.406+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f779002a930 tx=0x7f7790009260 comp rx=0 tx=0).stop 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2012674371 shutdown_connections 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf4154370 0x55aaf416ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf414f2e0 0x55aaf416b540 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2012674371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414e9a0 0x55aaf41490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2012674371 >> 172.21.15.129:0/2012674371 conn(0x55aaf414a010 msgr2=0x55aaf41489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2012674371 shutdown_connections 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2012674371 wait complete. 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- start start 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 Processor -- start 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf41caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf41cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41a8d40 con 0x55aaf4154370 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41aa1b0 con 0x55aaf414e9a0 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77a41a9410 con 0x55aaf414f2e0 2024-09-05T01:16:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.407+0000 7f77ad8b2740 10 client.0 _ll_register_callbacks cb 0x7f7798003d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38956/0 (socket says 172.21.15.129:38956) 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 -- 172.21.15.129:0/2232328381 learned_addr learned my addr 172.21.15.129:0/2232328381 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf41cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf41caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 -- 172.21.15.129:0/2232328381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 msgr2=0x55aaf41cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf41cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 -- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 msgr2=0x55aaf41caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf41caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 -- 172.21.15.129:0/2232328381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaf41d2a50 con 0x55aaf4154370 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a3fff640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf41cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77a90b5640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f779c009ff0 tx=0x7f779c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7905950756e1af0d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f7783fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c017070 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f779c00e6d0 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7790004240 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f779c041050 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77ad8b2740 10 client.15492 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.408+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55aaf41d3dd0 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7790004240 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f779c01b070 con 0x55aaf4154370 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 10 client.15492 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 20 client.15492 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41d4690 0x55aaf41d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 4 client.15492 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaf40b9d70 con 0x55aaf41d4690 2024-09-05T01:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.409+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41d4690 0x55aaf41d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.410+0000 7f7783fff640 10 client.15492 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.410+0000 7f77a88b4640 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41d4690 0x55aaf41d6ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06ff04a28550bbe server_cookie=fd8cbde9cfe179db in_seq=0 out_seq=0 2024-09-05T01:16:31.415 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:30 smithi183 ceph-mon[34589]: pgmap v386: 129 pgs: 129 active+clean; 227 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-09-05T01:16:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.422+0000 7f7783fff640 1 -- 172.21.15.129:0/2232328381 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aaf40b9d70 con 0x55aaf41d4690 2024-09-05T01:16:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.422+0000 7f7783fff640 10 client.15492 handle_command_reply: tid=0 2024-09-05T01:16:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.422+0000 7f77ad8b2740 1 client.15492 shutdown 2024-09-05T01:16:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 msgr2=0x55aaf41d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f779c009ff0 tx=0x7f779c00ff20 comp rx=0 tx=0).stop 2024-09-05T01:16:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 shutdown_connections 2024-09-05T01:16:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55aaf41d4690 0x55aaf41d6ab0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aaf4154370 0x55aaf41d2440 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55aaf414f2e0 0x55aaf41cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 --2- 172.21.15.129:0/2232328381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55aaf414e9a0 0x55aaf41caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 >> 172.21.15.129:0/2232328381 conn(0x55aaf414a010 msgr2=0x55aaf414b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f7783fff640 0 client.15492 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 shutdown_connections 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 1 -- 172.21.15.129:0/2232328381 wait complete. 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.423+0000 7f77ad8b2740 20 client.15492 trim_cache size 0 max 16384 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:16:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 msgr2=0x7f77940a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 0x7f77940a5190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f779c023000 tx=0x7f779c024000 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 msgr2=0x7f77a4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 0x7f77a4197d50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f779800e9c0 tx=0x7f779800ee80 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 shutdown_connections 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77940a2cd0 0x7f77940a5190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a41025f0 0x7f77a419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77a4101c30 0x7f77a4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 --2- 172.21.15.129:0/2188732227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77a4100a40 0x7f77a4197d50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 >> 172.21.15.129:0/2188732227 conn(0x7f77a40fc1e0 msgr2=0x7f77a40fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 shutdown_connections 2024-09-05T01:16:31.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:31.424+0000 7f77aab18640 1 -- 172.21.15.129:0/2188732227 wait complete. 2024-09-05T01:16:31.883 INFO:teuthology.orchestra.run.smithi129.stderr:1100000 of 3200000 tuples (34%) done (elapsed 36.34 s, remaining 69.38 s) 2024-09-05T01:16:32.656 INFO:teuthology.orchestra.run.smithi129.stderr:1200000 of 3200000 tuples (37%) done (elapsed 37.12 s, remaining 61.86 s) 2024-09-05T01:16:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:32 smithi129 ceph-mon[30979]: pgmap v387: 129 pgs: 129 active+clean; 227 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 344 KiB/s wr, 8 op/s 2024-09-05T01:16:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:32 smithi194 ceph-mon[34941]: pgmap v387: 129 pgs: 129 active+clean; 227 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 344 KiB/s wr, 8 op/s 2024-09-05T01:16:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:32 smithi183 ceph-mon[34589]: pgmap v387: 129 pgs: 129 active+clean; 227 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 344 KiB/s wr, 8 op/s 2024-09-05T01:16:33.686 INFO:teuthology.orchestra.run.smithi129.stderr:1300000 of 3200000 tuples (40%) done (elapsed 38.15 s, remaining 55.75 s) 2024-09-05T01:16:34.752 INFO:teuthology.orchestra.run.smithi129.stderr:1400000 of 3200000 tuples (43%) done (elapsed 39.21 s, remaining 50.42 s) 2024-09-05T01:16:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:34 smithi129 ceph-mon[30979]: pgmap v388: 129 pgs: 129 active+clean; 301 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-09-05T01:16:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:34 smithi194 ceph-mon[34941]: pgmap v388: 129 pgs: 129 active+clean; 301 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-09-05T01:16:35.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:34 smithi183 ceph-mon[34589]: pgmap v388: 129 pgs: 129 active+clean; 301 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 13 MiB/s wr, 20 op/s 2024-09-05T01:16:35.515 INFO:teuthology.orchestra.run.smithi129.stderr:1500000 of 3200000 tuples (46%) done (elapsed 39.98 s, remaining 45.31 s) 2024-09-05T01:16:36.567 INFO:teuthology.orchestra.run.smithi129.stderr:1600000 of 3200000 tuples (50%) done (elapsed 41.03 s, remaining 41.03 s) 2024-09-05T01:16:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:36 smithi129 ceph-mon[30979]: pgmap v389: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-05T01:16:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:36 smithi194 ceph-mon[34941]: pgmap v389: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-05T01:16:37.253 INFO:teuthology.orchestra.run.smithi129.stderr:1700000 of 3200000 tuples (53%) done (elapsed 41.71 s, remaining 36.81 s) 2024-09-05T01:16:37.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:36 smithi183 ceph-mon[34589]: pgmap v389: 129 pgs: 129 active+clean; 325 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 16 MiB/s wr, 17 op/s 2024-09-05T01:16:38.350 INFO:teuthology.orchestra.run.smithi129.stderr:1800000 of 3200000 tuples (56%) done (elapsed 42.81 s, remaining 33.30 s) 2024-09-05T01:16:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:38 smithi129 ceph-mon[30979]: pgmap v390: 129 pgs: 129 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-09-05T01:16:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:38 smithi194 ceph-mon[34941]: pgmap v390: 129 pgs: 129 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-09-05T01:16:39.380 INFO:teuthology.orchestra.run.smithi129.stderr:1900000 of 3200000 tuples (59%) done (elapsed 43.84 s, remaining 30.00 s) 2024-09-05T01:16:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:38 smithi183 ceph-mon[34589]: pgmap v390: 129 pgs: 129 active+clean; 369 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 22 op/s 2024-09-05T01:16:40.168 INFO:teuthology.orchestra.run.smithi129.stderr:2000000 of 3200000 tuples (62%) done (elapsed 44.63 s, remaining 26.78 s) 2024-09-05T01:16:41.192 INFO:teuthology.orchestra.run.smithi129.stderr:2100000 of 3200000 tuples (65%) done (elapsed 45.65 s, remaining 23.91 s) 2024-09-05T01:16:41.241 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:40 smithi129 ceph-mon[30979]: pgmap v391: 129 pgs: 129 active+clean; 431 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-05T01:16:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:40 smithi194 ceph-mon[34941]: pgmap v391: 129 pgs: 129 active+clean; 431 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-05T01:16:41.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:40 smithi183 ceph-mon[34589]: pgmap v391: 129 pgs: 129 active+clean; 431 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 33 MiB/s wr, 27 op/s 2024-09-05T01:16:42.254 INFO:teuthology.orchestra.run.smithi129.stderr:2200000 of 3200000 tuples (68%) done (elapsed 46.71 s, remaining 21.23 s) 2024-09-05T01:16:42.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:42 smithi183 ceph-mon[34589]: pgmap v392: 129 pgs: 129 active+clean; 439 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-09-05T01:16:42.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:42 smithi129 ceph-mon[30979]: pgmap v392: 129 pgs: 129 active+clean; 439 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-09-05T01:16:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:42 smithi194 ceph-mon[34941]: pgmap v392: 129 pgs: 129 active+clean; 439 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 35 MiB/s wr, 26 op/s 2024-09-05T01:16:43.037 INFO:teuthology.orchestra.run.smithi129.stderr:2300000 of 3200000 tuples (71%) done (elapsed 47.50 s, remaining 18.59 s) 2024-09-05T01:16:44.110 INFO:teuthology.orchestra.run.smithi129.stderr:2400000 of 3200000 tuples (75%) done (elapsed 48.57 s, remaining 16.19 s) 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3860239085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0104ef0 msgr2=0x7fedb01072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3860239085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0104ef0 0x7fedb01072e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7feda4009900 tx=0x7feda402a410 comp rx=0 tx=0).stop 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3860239085 shutdown_connections 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3860239085 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb0107820 0x7fedb0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3860239085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0104ef0 0x7fedb01072e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3860239085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb01025c0 0x7fedb01049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3860239085 >> 172.21.15.129:0/3860239085 conn(0x7fedb00fc1e0 msgr2=0x7fedb00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3860239085 shutdown_connections 2024-09-05T01:16:44.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.522+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3860239085 wait complete. 2024-09-05T01:16:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 Processor -- start 2024-09-05T01:16:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 -- start start 2024-09-05T01:16:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb01025c0 0x7fedb0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb0104ef0 0x7fedb01982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedb010d700 con 0x7fedb0107820 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedb010d580 con 0x7fedb0104ef0 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedb4bc4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fedb010d880 con 0x7fedb01025c0 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42164/0 (socket says 172.21.15.129:42164) 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 -- 172.21.15.129:0/3278010698 learned_addr learned my addr 172.21.15.129:0/3278010698 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedae59c640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb01025c0 0x7fedb0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedadd9b640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb0104ef0 0x7fedb01982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 -- 172.21.15.129:0/3278010698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb01025c0 msgr2=0x7fedb0197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb01025c0 0x7fedb0197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 -- 172.21.15.129:0/3278010698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb0104ef0 msgr2=0x7fedb01982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb0104ef0 0x7fedb01982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.523+0000 7fedaed9d640 1 -- 172.21.15.129:0/3278010698 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb019f980 con 0x7fedb0107820 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fedaed9d640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9d0 tx=0x7feda800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fed9f7fe640 1 -- 172.21.15.129:0/3278010698 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda800cd80 con 0x7fedb0107820 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fed9f7fe640 1 -- 172.21.15.129:0/3278010698 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda8005020 con 0x7fedb0107820 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb019fc70 con 0x7fedb0107820 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fed9f7fe640 1 -- 172.21.15.129:0/3278010698 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda80116d0 con 0x7fedb0107820 2024-09-05T01:16:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.524+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb01a0190 con 0x7fedb0107820 2024-09-05T01:16:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.526+0000 7fed9f7fe640 1 -- 172.21.15.129:0/3278010698 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 19) ==== 137885+0+0 (secure 0 0 0) 0x7feda80051d0 con 0x7fedb0107820 2024-09-05T01:16:44.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 0x7fed980a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fed9f7fe640 1 -- 172.21.15.129:0/3278010698 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7feda80cfa10 con 0x7fedb0107820 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedae59c640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 0x7fed980a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 Processor -- start 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 -- start start 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedae59c640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 0x7fed980a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feda00058c0 tx=0x7feda000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eeacada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 0x5648eead2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648eea2e2f0 con 0x5648eead0370 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648ee9d6270 con 0x5648eeacb2e0 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.527+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5648eea2a2b0 con 0x5648eeaca9a0 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 0x5648eead2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58280/0 (socket says 172.21.15.129:58280) 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedadd9b640 1 -- 172.21.15.129:0/2788145837 learned_addr learned my addr 172.21.15.129:0/2788145837 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedae59c640 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eeacada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 -- 172.21.15.129:0/2788145837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 msgr2=0x5648eeacada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eeacada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 -- 172.21.15.129:0/2788145837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 msgr2=0x5648eeacfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 -- 172.21.15.129:0/2788145837 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648ee9d5340 con 0x5648eead0370 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedadd9b640 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fedaed9d640 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 0x5648eead2830 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feda80ccfb0 tx=0x7feda809be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba554c1b82c41b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fed9cff9640 1 -- 172.21.15.129:0/2788145837 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda801f070 con 0x5648eead0370 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fed9cff9640 1 -- 172.21.15.129:0/2788145837 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda8005580 con 0x5648eead0370 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.528+0000 7fed9cff9640 1 -- 172.21.15.129:0/2788145837 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8097b00 con 0x5648eead0370 2024-09-05T01:16:44.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2788145837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 msgr2=0x5648eead2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 0x5648eead2830 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feda80ccfb0 tx=0x7feda809be80 comp rx=0 tx=0).stop 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2788145837 shutdown_connections 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eead0370 0x5648eead2830 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeacfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2788145837 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eeacada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2788145837 >> 172.21.15.129:0/2788145837 conn(0x5648eeac6010 msgr2=0x5648eeac6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2788145837 shutdown_connections 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.529+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2788145837 wait complete. 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 -- start start 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 Processor -- start 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eead4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eead0370 0x5648eeadc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648eead32e0 con 0x5648eeacb2e0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5648ee9b7290 con 0x5648eead0370 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5648eead3460 con 0x5648eeaca9a0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedb7a3c740 10 client.0 _ll_register_callbacks cb 0x5648ee9e8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42174/0 (socket says 172.21.15.129:42174) 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 -- 172.21.15.129:0/2865154251 learned_addr learned my addr 172.21.15.129:0/2865154251 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedae59c640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eead4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedaed9d640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eead0370 0x5648eeadc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 -- 172.21.15.129:0/2865154251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 msgr2=0x5648eead4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eead4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 -- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eead0370 msgr2=0x5648eeadc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eead0370 0x5648eeadc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 -- 172.21.15.129:0/2865154251 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ee741210 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fedadd9b640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feda402aa00 tx=0x7feda40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae9f0e41de81101 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.530+0000 7fed95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035040 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7feda4003b80 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda8005580 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feda404e050 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648ee80d620 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda8005580 con 0x5648eeacb2e0 2024-09-05T01:16:44.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7feda403c560 con 0x5648eeacb2e0 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 20 client.15504 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eeae07d0 0x5648eeae2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 4 client.15504 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.531+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648eea3ba20 con 0x5648eeae07d0 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.532+0000 7fedae59c640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eeae07d0 0x5648eeae2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.532+0000 7fed95ffb640 10 client.15504 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:44.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.532+0000 7fedae59c640 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eeae07d0 0x5648eeae2c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9091c2a560bf542d server_cookie=5eb6366b9e8666f8 in_seq=0 out_seq=0 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fed95ffb640 1 -- 172.21.15.129:0/2865154251 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648eea3ba20 con 0x5648eeae07d0 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fed95ffb640 10 client.15504 handle_command_reply: tid=0 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 client.15504 shutdown 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 msgr2=0x5648eead50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feda402aa00 tx=0x7feda40045e0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 shutdown_connections 2024-09-05T01:16:44.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eeae07d0 0x5648eeae2c30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eead0370 0x5648eeadc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeacb2e0 0x5648eead50a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fed95ffb640 0 client.15504 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/2865154251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeaca9a0 0x5648eead4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 >> 172.21.15.129:0/2865154251 conn(0x5648eeac6010 msgr2=0x5648eeac7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 shutdown_connections 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 1 -- 172.21.15.129:0/2865154251 wait complete. 2024-09-05T01:16:44.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.533+0000 7fedb7a3c740 20 client.15504 trim_cache size 0 max 16384 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 Processor -- start 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 -- start start 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeac6010 0x5648eeac50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeac7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01ad190 con 0x5648eeaca9a0 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01aaa60 con 0x5648eeacb2e0 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01a88b0 con 0x5648eeac6010 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedadd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42176/0 (socket says 172.21.15.129:42176) 2024-09-05T01:16:44.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedadd9b640 1 -- 172.21.15.129:0/3026386176 learned_addr learned my addr 172.21.15.129:0/3026386176 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.550+0000 7fedaed9d640 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeac7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedae59c640 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeac6010 0x5648eeac50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 -- 172.21.15.129:0/3026386176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeac6010 msgr2=0x5648eeac50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeac6010 0x5648eeac50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 -- 172.21.15.129:0/3026386176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 msgr2=0x5648eeac7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeac7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 -- 172.21.15.129:0/3026386176 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648eeac5e30 con 0x5648eeaca9a0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fedadd9b640 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feda403c980 tx=0x7feda4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d3ce975ff3aece server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fed97fff640 1 -- 172.21.15.129:0/3026386176 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4035070 con 0x5648eeaca9a0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fed97fff640 1 -- 172.21.15.129:0/3026386176 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda4005850 con 0x5648eeaca9a0 2024-09-05T01:16:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.551+0000 7fed97fff640 1 -- 172.21.15.129:0/3026386176 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4031940 con 0x5648eeaca9a0 2024-09-05T01:16:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- 172.21.15.129:0/3026386176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 msgr2=0x5648eeadc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feda403c980 tx=0x7feda4001e70 comp rx=0 tx=0).stop 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- 172.21.15.129:0/3026386176 shutdown_connections 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeacb2e0 0x5648eeac7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeaca9a0 0x5648eeadc030 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/3026386176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeac6010 0x5648eeac50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- 172.21.15.129:0/3026386176 >> 172.21.15.129:0/3026386176 conn(0x5648eeacfe90 msgr2=0x5648eeac49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- 172.21.15.129:0/3026386176 shutdown_connections 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- 172.21.15.129:0/3026386176 wait complete. 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 -- start start 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.552+0000 7fedb7a3c740 1 Processor -- start 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeaca9a0 0x5648eead9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeacb2e0 0x5648eeadad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01a8f20 con 0x5648eeac6010 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01aa390 con 0x5648eeaca9a0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fedb01a95f0 con 0x5648eeacb2e0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 10 client.0 _ll_register_callbacks cb 0x7feda4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42192/0 (socket says 172.21.15.129:42192) 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 -- 172.21.15.129:0/673960438 learned_addr learned my addr 172.21.15.129:0/673960438 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedaed9d640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeacb2e0 0x5648eeadad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedadd9b640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeaca9a0 0x5648eead9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 -- 172.21.15.129:0/673960438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeacb2e0 msgr2=0x5648eeadad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeacb2e0 0x5648eeadad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 -- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeaca9a0 msgr2=0x5648eead9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeaca9a0 0x5648eead9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 -- 172.21.15.129:0/673960438 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648eeadb310 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedae59c640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feda000f790 tx=0x7feda0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f8b7cf2df775ad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fed95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda0036060 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7feda001c9b0 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feda4005850 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.553+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feda003d050 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 10 client.15510 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5648eeb4b640 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feda4005850 con 0x5648eeac6010 2024-09-05T01:16:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7feda0029400 con 0x5648eeac6010 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 10 client.15510 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 20 client.15510 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eead0370 0x5648eeb4e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 4 client.15510 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5648eea35d70 con 0x5648eead0370 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.554+0000 7fedadd9b640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eead0370 0x5648eeb4e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.555+0000 7fedadd9b640 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eead0370 0x5648eeb4e020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acace85164df9a5f server_cookie=a5389c733a6ec8e8 in_seq=0 out_seq=0 2024-09-05T01:16:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.555+0000 7fed95ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.569+0000 7fed95ffb640 1 -- 172.21.15.129:0/673960438 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5648eea35d70 con 0x5648eead0370 2024-09-05T01:16:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.569+0000 7fed95ffb640 10 client.15510 handle_command_reply: tid=0 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.569+0000 7fedb7a3c740 1 client.15510 shutdown 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.569+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 msgr2=0x5648eead97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.569+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feda000f790 tx=0x7feda0004730 comp rx=0 tx=0).stop 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 shutdown_connections 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5648eead0370 0x5648eeb4e020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fed95ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5648eeacb2e0 0x5648eeadad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5648eeaca9a0 0x5648eead9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 --2- 172.21.15.129:0/673960438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648eeac6010 0x5648eead97b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 >> 172.21.15.129:0/673960438 conn(0x5648eeacfe90 msgr2=0x5648eeac51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 shutdown_connections 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 1 -- 172.21.15.129:0/673960438 wait complete. 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb7a3c740 20 client.15510 trim_cache size 0 max 16384 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:16:44.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:16:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 msgr2=0x7fed980a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 0x7fed980a5210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feda00058c0 tx=0x7feda000a2d0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 msgr2=0x7fedb019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:44.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7feda800e9d0 tx=0x7feda800ee90 comp rx=0 tx=0).stop 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 shutdown_connections 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed980a2d50 0x7fed980a5210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedb0107820 0x7fedb019f310 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fedb0104ef0 0x7fedb01982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 --2- 172.21.15.129:0/3278010698 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fedb01025c0 0x7fedb0197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 >> 172.21.15.129:0/3278010698 conn(0x7fedb00fc1e0 msgr2=0x7fedb0102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.570+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 shutdown_connections 2024-09-05T01:16:44.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:44.571+0000 7fedb4bc4640 1 -- 172.21.15.129:0/3278010698 wait complete. 2024-09-05T01:16:45.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:44 smithi183 ceph-mon[34589]: pgmap v393: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 51 MiB/s wr, 37 op/s 2024-09-05T01:16:45.179 INFO:teuthology.orchestra.run.smithi129.stderr:2500000 of 3200000 tuples (78%) done (elapsed 49.64 s, remaining 13.90 s) 2024-09-05T01:16:45.241 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:44 smithi129 ceph-mon[30979]: pgmap v393: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 51 MiB/s wr, 37 op/s 2024-09-05T01:16:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:44 smithi194 ceph-mon[34941]: pgmap v393: 129 pgs: 129 active+clean; 529 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 51 MiB/s wr, 37 op/s 2024-09-05T01:16:45.906 INFO:teuthology.orchestra.run.smithi129.stderr:2600000 of 3200000 tuples (81%) done (elapsed 50.37 s, remaining 11.62 s) 2024-09-05T01:16:46.971 INFO:teuthology.orchestra.run.smithi129.stderr:2700000 of 3200000 tuples (84%) done (elapsed 51.43 s, remaining 9.52 s) 2024-09-05T01:16:47.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:46 smithi183 ceph-mon[34589]: pgmap v394: 129 pgs: 129 active+clean; 569 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 27 op/s 2024-09-05T01:16:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:46 smithi129 ceph-mon[30979]: pgmap v394: 129 pgs: 129 active+clean; 569 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 27 op/s 2024-09-05T01:16:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:46 smithi194 ceph-mon[34941]: pgmap v394: 129 pgs: 129 active+clean; 569 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 27 op/s 2024-09-05T01:16:47.732 INFO:teuthology.orchestra.run.smithi129.stderr:2800000 of 3200000 tuples (87%) done (elapsed 52.19 s, remaining 7.46 s) 2024-09-05T01:16:48.798 INFO:teuthology.orchestra.run.smithi129.stderr:2900000 of 3200000 tuples (90%) done (elapsed 53.26 s, remaining 5.51 s) 2024-09-05T01:16:49.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:48 smithi183 ceph-mon[34589]: pgmap v395: 129 pgs: 129 active+clean; 603 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 MiB/s wr, 34 op/s 2024-09-05T01:16:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:48 smithi129 ceph-mon[30979]: pgmap v395: 129 pgs: 129 active+clean; 603 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 MiB/s wr, 34 op/s 2024-09-05T01:16:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:48 smithi194 ceph-mon[34941]: pgmap v395: 129 pgs: 129 active+clean; 603 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 46 MiB/s wr, 34 op/s 2024-09-05T01:16:49.821 INFO:teuthology.orchestra.run.smithi129.stderr:3000000 of 3200000 tuples (93%) done (elapsed 54.28 s, remaining 3.62 s) 2024-09-05T01:16:50.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:49 smithi183 ceph-mon[34589]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T01:16:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:49 smithi129 ceph-mon[30979]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T01:16:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:49 smithi194 ceph-mon[34941]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T01:16:50.610 INFO:teuthology.orchestra.run.smithi129.stderr:3100000 of 3200000 tuples (96%) done (elapsed 55.07 s, remaining 1.78 s) 2024-09-05T01:16:51.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:50 smithi183 ceph-mon[34589]: pgmap v396: 129 pgs: 129 active+clean; 661 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 MiB/s wr, 38 op/s 2024-09-05T01:16:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:50 smithi129 ceph-mon[30979]: pgmap v396: 129 pgs: 129 active+clean; 661 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 MiB/s wr, 38 op/s 2024-09-05T01:16:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:50 smithi194 ceph-mon[34941]: pgmap v396: 129 pgs: 129 active+clean; 661 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 50 MiB/s wr, 38 op/s 2024-09-05T01:16:51.776 INFO:teuthology.orchestra.run.smithi129.stderr:3200000 of 3200000 tuples (100%) done (elapsed 56.24 s, remaining 0.00 s) 2024-09-05T01:16:51.848 INFO:teuthology.orchestra.run.smithi129.stderr:vacuuming... 2024-09-05T01:16:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:52 smithi183 ceph-mon[34589]: pgmap v397: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 42 MiB/s wr, 33 op/s 2024-09-05T01:16:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:52 smithi129 ceph-mon[30979]: pgmap v397: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 42 MiB/s wr, 33 op/s 2024-09-05T01:16:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:52 smithi194 ceph-mon[34941]: pgmap v397: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 42 MiB/s wr, 33 op/s 2024-09-05T01:16:55.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:54 smithi183 ceph-mon[34589]: pgmap v398: 129 pgs: 129 active+clean; 759 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:16:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:54 smithi129 ceph-mon[30979]: pgmap v398: 129 pgs: 129 active+clean; 759 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:16:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:54 smithi194 ceph-mon[34941]: pgmap v398: 129 pgs: 129 active+clean; 759 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:16:57.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:56 smithi183 ceph-mon[34589]: pgmap v399: 129 pgs: 129 active+clean; 771 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 35 op/s 2024-09-05T01:16:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:56 smithi129 ceph-mon[30979]: pgmap v399: 129 pgs: 129 active+clean; 771 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 35 op/s 2024-09-05T01:16:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:56 smithi194 ceph-mon[34941]: pgmap v399: 129 pgs: 129 active+clean; 771 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 44 MiB/s wr, 35 op/s 2024-09-05T01:16:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 -- 172.21.15.129:0/775699925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e800ffbc0 msgr2=0x7f3e80100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 --2- 172.21.15.129:0/775699925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e800ffbc0 0x7f3e80100040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009930 tx=0x7f3e7402a420 comp rx=0 tx=0).stop 2024-09-05T01:16:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 -- 172.21.15.129:0/775699925 shutdown_connections 2024-09-05T01:16:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 --2- 172.21.15.129:0/775699925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e80100580 0x7f3e8010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 --2- 172.21.15.129:0/775699925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e800ffbc0 0x7f3e80100040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 --2- 172.21.15.129:0/775699925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80103150 0x7f3e80103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 -- 172.21.15.129:0/775699925 >> 172.21.15.129:0/775699925 conn(0x7f3e800fb930 msgr2=0x7f3e800fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 -- 172.21.15.129:0/775699925 shutdown_connections 2024-09-05T01:16:57.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.671+0000 7f3e8495f640 1 -- 172.21.15.129:0/775699925 wait complete. 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.672+0000 7f3e8495f640 1 Processor -- start 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.672+0000 7f3e8495f640 1 -- start start 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.672+0000 7f3e8495f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 0x7f3e80197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 0x7f3e801982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010d680 con 0x7f3e80103150 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010d500 con 0x7f3e80100580 2024-09-05T01:16:57.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e8010d800 con 0x7f3e800ffbc0 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45604/0 (socket says 172.21.15.129:45604) 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 -- 172.21.15.129:0/2673783503 learned_addr learned my addr 172.21.15.129:0/2673783503 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 0x7f3e80197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 0x7f3e801982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 -- 172.21.15.129:0/2673783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 msgr2=0x7f3e80197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 0x7f3e80197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 -- 172.21.15.129:0/2673783503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 msgr2=0x7f3e801982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 0x7f3e801982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 -- 172.21.15.129:0/2673783503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e8019f980 con 0x7f3e80103150 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 0x7f3e80197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 0x7f3e801982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:16:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e7ed9d640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009fd0 tx=0x7f3e7800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e6f7fe640 1 -- 172.21.15.129:0/2673783503 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78017070 con 0x7f3e80103150 2024-09-05T01:16:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e8019fc70 con 0x7f3e80103150 2024-09-05T01:16:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e6f7fe640 1 -- 172.21.15.129:0/2673783503 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e78003e20 con 0x7f3e80103150 2024-09-05T01:16:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e6f7fe640 1 -- 172.21.15.129:0/2673783503 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e78010460 con 0x7f3e80103150 2024-09-05T01:16:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.673+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e801a0190 con 0x7f3e80103150 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.676+0000 7f3e877d5740 1 Processor -- start 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e6f7fe640 1 -- 172.21.15.129:0/2673783503 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3e78003970 con 0x7f3e80103150 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 -- start start 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5599ca9202f0 con 0x5599ca9c2370 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599ca8c8270 con 0x5599ca9bc9a0 2024-09-05T01:16:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e877d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599ca91c2b0 con 0x5599ca9bd2e0 2024-09-05T01:16:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45606/0 (socket says 172.21.15.129:45606) 2024-09-05T01:16:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.677+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3272263195 learned_addr learned my addr 172.21.15.129:0/3272263195 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9bcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e6f7fe640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 0x7f3e680a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3272263195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 msgr2=0x5599ca9c1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3272263195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 msgr2=0x5599ca9bcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9bcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3272263195 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ca8c7340 con 0x5599ca9c2370 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7ed9d640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 0x7f3e680a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e6f7fe640 1 -- 172.21.15.129:0/2673783503 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3e780cf390 con 0x7f3e80103150 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e740314e0 tx=0x7f3e74005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70af0768a3645989 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e6cff9640 1 -- 172.21.15.129:0/3272263195 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e74035050 con 0x5599ca9c2370 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e6cff9640 1 -- 172.21.15.129:0/3272263195 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e74005040 con 0x5599ca9c2370 2024-09-05T01:16:57.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.678+0000 7f3e6cff9640 1 -- 172.21.15.129:0/3272263195 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740048e0 con 0x5599ca9c2370 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 -- 172.21.15.129:0/3272263195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 msgr2=0x5599ca9c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e7ed9d640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 0x7f3e680a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809b4e0 tx=0x7f3e780663b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3e740314e0 tx=0x7f3e74005b50 comp rx=0 tx=0).stop 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 -- 172.21.15.129:0/3272263195 shutdown_connections 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9c4830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3272263195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.679+0000 7f3e877d5740 1 -- 172.21.15.129:0/3272263195 >> 172.21.15.129:0/3272263195 conn(0x5599ca9b8010 msgr2=0x5599ca9b8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- 172.21.15.129:0/3272263195 shutdown_connections 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- 172.21.15.129:0/3272263195 wait complete. 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- start start 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 Processor -- start 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9c2370 0x5599ca9ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5599ca9c52e0 con 0x5599ca9bc9a0 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5599ca8a9290 con 0x5599ca9c2370 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5599ca9c5460 con 0x5599ca9bd2e0 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.680+0000 7f3e877d5740 10 client.0 _ll_register_callbacks cb 0x5599ca8da140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45612/0 (socket says 172.21.15.129:45612) 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3741995739 learned_addr learned my addr 172.21.15.129:0/3741995739 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9c2370 0x5599ca9ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3741995739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 msgr2=0x5599ca9c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9c2370 msgr2=0x5599ca9ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9c2370 0x5599ca9ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/3741995739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ca633210 con 0x5599ca9bc9a0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e74004650 tx=0x7f3e74009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3d3bff736a7a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7405a060 con 0x5599ca9bc9a0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3e7404bda0 con 0x5599ca9bc9a0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e74005040 con 0x5599ca9bc9a0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e74061050 con 0x5599ca9bc9a0 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e877d5740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:57.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.681+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599ca6ff620 con 0x5599ca9bc9a0 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e74005040 con 0x5599ca9bc9a0 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3e7403a600 con 0x5599ca9bc9a0 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 20 client.15522 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599ca9d2480 0x5599ca9d4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 4 client.15522 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599ca92da20 con 0x5599ca9d2480 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599ca9d2480 0x5599ca9d4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599ca9d2480 0x5599ca9d4870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa83ae1fcc683b4 server_cookie=beef51e9a81b3caf in_seq=0 out_seq=0 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.682+0000 7f3e65ffb640 10 client.15522 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e65ffb640 1 -- 172.21.15.129:0/3741995739 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599ca92da20 con 0x5599ca9d2480 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e65ffb640 10 client.15522 handle_command_reply: tid=0 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 client.15522 shutdown 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 msgr2=0x5599ca9c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3e74004650 tx=0x7f3e74009260 comp rx=0 tx=0).stop 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 shutdown_connections 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599ca9d2480 0x5599ca9d4870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9c2370 0x5599ca9ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 --2- 172.21.15.129:0/3741995739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c6c00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 >> 172.21.15.129:0/3741995739 conn(0x5599ca9b8010 msgr2=0x5599ca9b9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e65ffb640 0 client.15522 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 shutdown_connections 2024-09-05T01:16:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.683+0000 7f3e877d5740 1 -- 172.21.15.129:0/3741995739 wait complete. 2024-09-05T01:16:57.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.684+0000 7f3e877d5740 20 client.15522 trim_cache size 0 max 16384 2024-09-05T01:16:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 Processor -- start 2024-09-05T01:16:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 -- start start 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 0x5599ca9b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801aae50 con 0x5599ca9bd2e0 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801a8720 con 0x5599ca9bc9a0 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.702+0000 7f3e877d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801a6570 con 0x5599ca9c2370 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45618/0 (socket says 172.21.15.129:45618) 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 -- 172.21.15.129:0/1616454959 learned_addr learned my addr 172.21.15.129:0/1616454959 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 0x5599ca9b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 -- 172.21.15.129:0/1616454959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 msgr2=0x5599ca9b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 0x5599ca9b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 -- 172.21.15.129:0/1616454959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 msgr2=0x5599ca9c70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 -- 172.21.15.129:0/1616454959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ca9bed80 con 0x5599ca9bd2e0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 0x5599ca9b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e800695a0 tx=0x7f3e70005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fea596a85a9b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e67fff640 1 -- 172.21.15.129:0/1616454959 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700130d0 con 0x5599ca9bd2e0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e67fff640 1 -- 172.21.15.129:0/1616454959 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e70013270 con 0x5599ca9bd2e0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.703+0000 7f3e67fff640 1 -- 172.21.15.129:0/1616454959 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e700117c0 con 0x5599ca9bd2e0 2024-09-05T01:16:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- 172.21.15.129:0/1616454959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 msgr2=0x5599ca9ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e800695a0 tx=0x7f3e70005880 comp rx=0 tx=0).stop 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- 172.21.15.129:0/1616454959 shutdown_connections 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9c2370 0x5599ca9b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9bd2e0 0x5599ca9ce110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1616454959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9c70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- 172.21.15.129:0/1616454959 >> 172.21.15.129:0/1616454959 conn(0x5599ca9b8010 msgr2=0x5599ca9c69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- 172.21.15.129:0/1616454959 shutdown_connections 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- 172.21.15.129:0/1616454959 wait complete. 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.704+0000 7f3e877d5740 1 -- start start 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 Processor -- start 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801a6be0 con 0x5599ca9c2370 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801a8050 con 0x5599ca9bc9a0 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3e801a72b0 con 0x5599ca9bd2e0 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e877d5740 10 client.0 _ll_register_callbacks cb 0x7f3e74032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45634/0 (socket says 172.21.15.129:45634) 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/1283644579 learned_addr learned my addr 172.21.15.129:0/1283644579 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/1283644579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 msgr2=0x5599ca9cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 msgr2=0x5599ca9cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 -- 172.21.15.129:0/1283644579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ca9cd4a0 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e7dd9b640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e7403aa00 tx=0x7f3e7402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab3baf3f0a1b33b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.705+0000 7f3e65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7405a040 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3e74031760 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e70013270 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e74035040 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 10 client.15528 fetch_fsmap learned FSMap version 27 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599caa3d200 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e70013270 con 0x5599ca9c2370 2024-09-05T01:16:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3e74004950 con 0x5599ca9c2370 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 10 client.15528 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 20 client.15528 populate_metadata read hostname 'smithi129' 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599caa3db50 0x5599caa3ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 4 client.15528 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5599ca927d70 con 0x5599caa3db50 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.706+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599caa3db50 0x5599caa3ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.707+0000 7f3e7e59c640 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599caa3db50 0x5599caa3ff70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704cf5af57f98eab server_cookie=c54fd427e2a6e4cc in_seq=0 out_seq=0 2024-09-05T01:16:57.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.707+0000 7f3e65ffb640 10 client.15528 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e65ffb640 1 -- 172.21.15.129:0/1283644579 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5599ca927d70 con 0x5599caa3db50 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e65ffb640 10 client.15528 handle_command_reply: tid=0 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 client.15528 shutdown 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 msgr2=0x5599ca9cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3e7403aa00 tx=0x7f3e7402a420 comp rx=0 tx=0).stop 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 shutdown_connections 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5599caa3db50 0x5599caa3ff70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599ca9c2370 0x5599ca9cce90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5599ca9bd2e0 0x5599ca9cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e65ffb640 0 client.15528 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 --2- 172.21.15.129:0/1283644579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5599ca9bc9a0 0x5599ca9cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 >> 172.21.15.129:0/1283644579 conn(0x5599ca9b8010 msgr2=0x5599ca9b8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 shutdown_connections 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 1 -- 172.21.15.129:0/1283644579 wait complete. 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.730+0000 7f3e877d5740 20 client.15528 trim_cache size 0 max 16384 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:16:57.732 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:16:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:16:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:16:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 msgr2=0x7f3e680a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.733 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 0x7f3e680a50f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3e7809b4e0 tx=0x7f3e780663b0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 msgr2=0x7f3e8019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009fd0 tx=0x7f3e7800b260 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 shutdown_connections 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3e680a2c30 0x7f3e680a50f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3e80103150 0x7f3e8019f310 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e80100580 0x7f3e801982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 --2- 172.21.15.129:0/2673783503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3e800ffbc0 0x7f3e80197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 >> 172.21.15.129:0/2673783503 conn(0x7f3e800fb930 msgr2=0x7f3e801062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 shutdown_connections 2024-09-05T01:16:57.734 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:16:57.731+0000 7f3e8495f640 1 -- 172.21.15.129:0/2673783503 wait complete. 2024-09-05T01:16:59.061 INFO:teuthology.orchestra.run.smithi129.stderr:creating primary keys... 2024-09-05T01:16:59.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:16:58 smithi183 ceph-mon[34589]: pgmap v400: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 MiB/s wr, 39 op/s 2024-09-05T01:16:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:16:58 smithi129 ceph-mon[30979]: pgmap v400: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 MiB/s wr, 39 op/s 2024-09-05T01:16:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:16:58 smithi194 ceph-mon[34941]: pgmap v400: 129 pgs: 129 active+clean; 772 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 MiB/s wr, 39 op/s 2024-09-05T01:17:01.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:00 smithi183 ceph-mon[34589]: pgmap v401: 129 pgs: 129 active+clean; 780 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:17:01.241 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:00 smithi129 ceph-mon[30979]: pgmap v401: 129 pgs: 129 active+clean; 780 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:17:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:00 smithi194 ceph-mon[34941]: pgmap v401: 129 pgs: 129 active+clean; 780 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 57 MiB/s wr, 45 op/s 2024-09-05T01:17:03.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:02 smithi183 ceph-mon[34589]: pgmap v402: 129 pgs: 129 active+clean; 784 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-09-05T01:17:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:02 smithi129 ceph-mon[30979]: pgmap v402: 129 pgs: 129 active+clean; 784 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-09-05T01:17:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:02 smithi194 ceph-mon[34941]: pgmap v402: 129 pgs: 129 active+clean; 784 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 48 MiB/s wr, 37 op/s 2024-09-05T01:17:05.035 INFO:teuthology.orchestra.run.smithi129.stderr:done in 69.57 s (drop tables 0.01 s, create tables 0.05 s, client-side generate 56.33 s, vacuum 7.21 s, primary keys 5.97 s). 2024-09-05T01:17:05.039 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-05T01:17:05.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:04 smithi183 ceph-mon[34589]: pgmap v403: 129 pgs: 129 active+clean; 869 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 MiB/s wr, 57 op/s 2024-09-05T01:17:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:starting vacuum...end. 2024-09-05T01:17:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:04 smithi129 ceph-mon[30979]: pgmap v403: 129 pgs: 129 active+clean; 869 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 MiB/s wr, 57 op/s 2024-09-05T01:17:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:04 smithi194 ceph-mon[34941]: pgmap v403: 129 pgs: 129 active+clean; 869 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 63 MiB/s wr, 57 op/s 2024-09-05T01:17:07.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:06 smithi183 ceph-mon[34589]: pgmap v404: 129 pgs: 129 active+clean; 893 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 MiB/s wr, 46 op/s 2024-09-05T01:17:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:06 smithi129 ceph-mon[30979]: pgmap v404: 129 pgs: 129 active+clean; 893 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 MiB/s wr, 46 op/s 2024-09-05T01:17:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:06 smithi194 ceph-mon[34941]: pgmap v404: 129 pgs: 129 active+clean; 893 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 50 MiB/s wr, 46 op/s 2024-09-05T01:17:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:08 smithi129 ceph-mon[30979]: pgmap v405: 129 pgs: 129 active+clean; 951 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 79 op/s 2024-09-05T01:17:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:08 smithi194 ceph-mon[34941]: pgmap v405: 129 pgs: 129 active+clean; 951 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 79 op/s 2024-09-05T01:17:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:08 smithi183 ceph-mon[34589]: pgmap v405: 129 pgs: 129 active+clean; 951 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 53 MiB/s wr, 79 op/s 2024-09-05T01:17:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 5.0 s, 1651.4 tps, lat 56.928 ms stddev 64.579 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:09 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:09 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:09 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 -- 172.21.15.129:0/2034426224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 msgr2=0x7f1f780984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/2034426224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f780984e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009900 tx=0x7f1f7002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 -- 172.21.15.129:0/2034426224 shutdown_connections 2024-09-05T01:17:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/2034426224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 0x7f1f7809d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/2034426224 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 0x7f1f7809ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/2034426224 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f780984e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 -- 172.21.15.129:0/2034426224 >> 172.21.15.129:0/2034426224 conn(0x7f1f7808fcb0 msgr2=0x7f1f78092110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.867+0000 7f1f7f59e640 1 -- 172.21.15.129:0/2034426224 shutdown_connections 2024-09-05T01:17:10.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.868+0000 7f1f7f59e640 1 -- 172.21.15.129:0/2034426224 wait complete. 2024-09-05T01:17:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 Processor -- start 2024-09-05T01:17:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 -- start start 2024-09-05T01:17:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 0x7f1f781324a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 0x7f1f78139510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f78095240 con 0x7f1f78098a20 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f780950c0 con 0x7f1f7809b350 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7f59e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f780953c0 con 0x7f1f780960f0 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60984/0 (socket says 172.21.15.129:60984) 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7e59c640 1 -- 172.21.15.129:0/470108550 learned_addr learned my addr 172.21.15.129:0/470108550 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 0x7f1f781324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.869+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 0x7f1f78139510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 -- 172.21.15.129:0/470108550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 msgr2=0x7f1f78139510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 0x7f1f78139510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 -- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 msgr2=0x7f1f781324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 0x7f1f781324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 -- 172.21.15.129:0/470108550 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f78139b80 con 0x7f1f780960f0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7e59c640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f700314e0 tx=0x7f1f70003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f6f7fe640 1 -- 172.21.15.129:0/470108550 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7003f070 con 0x7f1f780960f0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f6f7fe640 1 -- 172.21.15.129:0/470108550 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f70031c40 con 0x7f1f780960f0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f78139e10 con 0x7f1f780960f0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7813a270 con 0x7f1f780960f0 2024-09-05T01:17:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.870+0000 7f1f6f7fe640 1 -- 172.21.15.129:0/470108550 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70035030 con 0x7f1f780960f0 2024-09-05T01:17:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.874+0000 7f1f6f7fe640 1 -- 172.21.15.129:0/470108550 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1f70042070 con 0x7f1f780960f0 2024-09-05T01:17:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.874+0000 7f1f87c0d740 1 Processor -- start 2024-09-05T01:17:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 -- start start 2024-09-05T01:17:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a553711f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553716f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553719a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a5536752f0 con 0x55a5537124b0 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a55361d270 con 0x55a553717540 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a5536712b0 con 0x55a553711b70 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553716f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553716f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59366/0 (socket says 172.21.15.129:59366) 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f7e59c640 1 -- 172.21.15.129:0/3947561715 learned_addr learned my addr 172.21.15.129:0/3947561715 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a553711f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.875+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553719a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3947561715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 msgr2=0x55a553719a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553719a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3947561715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 msgr2=0x55a553716f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553716f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3947561715 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55361c340 con 0x55a553711b70 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f6f7fe640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 0x7f1f680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7e59c640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 0x7f1f680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a553711f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f80050280 tx=0x7f1f8006b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7befc03e02af134b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f6f7fe640 1 -- 172.21.15.129:0/470108550 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1f700efb30 con 0x7f1f780960f0 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.876+0000 7f1f6cff9640 1 -- 172.21.15.129:0/3947561715 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80069e10 con 0x55a553711b70 2024-09-05T01:17:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.877+0000 7f1f6cff9640 1 -- 172.21.15.129:0/3947561715 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.877+0000 7f1f6cff9640 1 -- 172.21.15.129:0/3947561715 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80088960 con 0x55a553711b70 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3947561715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 msgr2=0x55a553711f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a553711f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f80050280 tx=0x7f1f8006b420 comp rx=0 tx=0).stop 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f7e59c640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 0x7f1f680a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1f7002a9b0 tx=0x7f1f700f3570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3947561715 shutdown_connections 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553719a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553716f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3947561715 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a553711f70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3947561715 >> 172.21.15.129:0/3947561715 conn(0x55a55370d1e0 msgr2=0x55a55370d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3947561715 shutdown_connections 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3947561715 wait complete. 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.878+0000 7f1f87c0d740 1 -- start start 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 Processor -- start 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a55371c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a553723330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:32768/0 (socket says 172.21.15.129:32768) 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3274765774 learned_addr learned my addr 172.21.15.129:0/3274765774 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a55371a510 con 0x55a553717540 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a5535fe290 con 0x55a5537124b0 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a55371a690 con 0x55a553711b70 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.879+0000 7f1f87c0d740 10 client.0 _ll_register_callbacks cb 0x55a55362f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a553723330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a55371c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 msgr2=0x55a55371c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a55371c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3274765774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 msgr2=0x55a553723330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a553723330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a553388210 con 0x55a553711b70 2024-09-05T01:17:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a55371c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006bbe0 tx=0x7f1f80063770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0753471fbba4125 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8007f640 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.880+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1f8007f7e0 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f8007f9e0 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 10 client.25066 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a553454620 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f1f800978d0 con 0x55a553711b70 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 10 client.25066 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 10 client.25066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 20 client.25066 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553729d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 4 client.25066 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.881+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a553682a20 con 0x55a553727950 2024-09-05T01:17:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.882+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553729d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.882+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553729d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f8aa9baf3030cb server_cookie=541f416b4f844876 in_seq=0 out_seq=0 2024-09-05T01:17:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.882+0000 7f1f65ffb640 10 client.25066 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:10.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f65ffb640 1 -- 172.21.15.129:0/3274765774 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a553682a20 con 0x55a553727950 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f65ffb640 10 client.25066 handle_command_reply: tid=0 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 client.25066 shutdown 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 msgr2=0x55a55371be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006bbe0 tx=0x7f1f80063770 comp rx=0 tx=0).stop 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 shutdown_connections 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553729d40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f65ffb640 0 client.25066 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a553723330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a55371c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/3274765774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371be20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.883+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 >> 172.21.15.129:0/3274765774 conn(0x55a55370d1e0 msgr2=0x55a55370e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.884+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 shutdown_connections 2024-09-05T01:17:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.884+0000 7f1f87c0d740 1 -- 172.21.15.129:0/3274765774 wait complete. 2024-09-05T01:17:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.884+0000 7f1f87c0d740 20 client.25066 trim_cache size 0 max 16384 2024-09-05T01:17:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.911+0000 7f1f87c0d740 1 Processor -- start 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.911+0000 7f1f87c0d740 1 -- start start 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a553723250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a55372e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f60012690 con 0x55a553717540 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6000ff60 con 0x55a5537124b0 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f87c0d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6000ddb0 con 0x55a553711b70 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:32784/0 (socket says 172.21.15.129:32784) 2024-09-05T01:17:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2620969742 learned_addr learned my addr 172.21.15.129:0/2620969742 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a55372e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a553723250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2620969742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 msgr2=0x55a553723250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a553723250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2620969742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 msgr2=0x55a55372e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a55372e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.912+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2620969742 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a55372a0b0 con 0x55a553711b70 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.913+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f80051220 tx=0x7f1f800506b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b485ab2a95490da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.913+0000 7f1f67fff640 1 -- 172.21.15.129:0/2620969742 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80088470 con 0x55a553711b70 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.913+0000 7f1f67fff640 1 -- 172.21.15.129:0/2620969742 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.913+0000 7f1f67fff640 1 -- 172.21.15.129:0/2620969742 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80068b70 con 0x55a553711b70 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2620969742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 msgr2=0x55a55371c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f80051220 tx=0x7f1f800506b0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2620969742 shutdown_connections 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a553717540 0x55a55372e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a5537124b0 0x55a553723250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2620969742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a55371c240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.914+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2620969742 >> 172.21.15.129:0/2620969742 conn(0x55a55370d1e0 msgr2=0x55a55370e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.915+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2620969742 shutdown_connections 2024-09-05T01:17:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.915+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2620969742 wait complete. 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.915+0000 7f1f87c0d740 1 -- start start 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.915+0000 7f1f87c0d740 1 Processor -- start 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f87c0d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553720de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553720de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553720de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59386/0 (socket says 172.21.15.129:59386) 2024-09-05T01:17:10.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.916+0000 7f1f7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:32788/0 (socket says 172.21.15.129:32788) 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f7dd9b640 1 -- 172.21.15.129:0/2363364053 learned_addr learned my addr 172.21.15.129:0/2363364053 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553721df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6000e420 con 0x55a5537124b0 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6000f890 con 0x55a553717540 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6000eaf0 con 0x55a553711b70 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f87c0d740 10 client.0 _ll_register_callbacks cb 0x7f1f700f3440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.917+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553721df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 msgr2=0x55a553721df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553721df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2363364053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 msgr2=0x55a553720de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553720de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a553722400 con 0x55a553711b70 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f7ed9d640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f80051220 tx=0x7f1f80068340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af7252b23781984 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f800b3070 con 0x55a553711b70 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f1f8007fd80 con 0x55a553711b70 2024-09-05T01:17:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.918+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.920+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f1f80097420 con 0x55a553711b70 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.920+0000 7f1f87c0d740 10 client.25072 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.920+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a553792620 con 0x55a553711b70 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f80083070 con 0x55a553711b70 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f1f800691c0 con 0x55a553711b70 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 10 client.25072 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 10 client.25072 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 20 client.25072 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553794d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 4 client.25072 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:17:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a55367cd70 con 0x55a553727950 2024-09-05T01:17:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.921+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553794d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.923+0000 7f1f7dd9b640 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553794d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc22de04b9df648 server_cookie=a34aaf276b3a4af5 in_seq=0 out_seq=0 2024-09-05T01:17:10.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.923+0000 7f1f65ffb640 10 client.25072 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f65ffb640 1 -- 172.21.15.129:0/2363364053 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a55367cd70 con 0x55a553727950 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f65ffb640 10 client.25072 handle_command_reply: tid=0 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 client.25072 shutdown 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 msgr2=0x55a5537208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f80051220 tx=0x7f1f80068340 comp rx=0 tx=0).stop 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 shutdown_connections 2024-09-05T01:17:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a553727950 0x55a553794d20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a553717540 0x55a553721df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5537124b0 0x55a553720de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 --2- 172.21.15.129:0/2363364053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a553711b70 0x55a5537208a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 >> 172.21.15.129:0/2363364053 conn(0x55a55370d1e0 msgr2=0x55a55370ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f65ffb640 0 client.25072 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 shutdown_connections 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.957+0000 7f1f87c0d740 1 -- 172.21.15.129:0/2363364053 wait complete. 2024-09-05T01:17:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.958+0000 7f1f87c0d740 20 client.25072 trim_cache size 0 max 16384 2024-09-05T01:17:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:17:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:17:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:17:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.958+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 msgr2=0x7f1f680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.958+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 0x7f1f680a50d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1f7002a9b0 tx=0x7f1f700f3570 comp rx=0 tx=0).stop 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 msgr2=0x7f1f78131f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1f700314e0 tx=0x7f1f70003c10 comp rx=0 tx=0).stop 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 shutdown_connections 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2c10 0x7f1f680a50d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7809b350 0x7f1f78139510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f78098a20 0x7f1f781324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 --2- 172.21.15.129:0/470108550 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f780960f0 0x7f1f78131f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 >> 172.21.15.129:0/470108550 conn(0x7f1f7808fcb0 msgr2=0x7f1f78096cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 shutdown_connections 2024-09-05T01:17:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:10.959+0000 7f1f7f59e640 1 -- 172.21.15.129:0/470108550 wait complete. 2024-09-05T01:17:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:10 smithi129 ceph-mon[30979]: pgmap v406: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 MiB/s wr, 79 op/s 2024-09-05T01:17:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:17:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:17:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:10 smithi194 ceph-mon[34941]: pgmap v406: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 MiB/s wr, 79 op/s 2024-09-05T01:17:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:17:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:17:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:10 smithi183 ceph-mon[34589]: pgmap v406: 129 pgs: 129 active+clean; 960 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 49 MiB/s wr, 79 op/s 2024-09-05T01:17:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:17:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:17:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:17:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:12 smithi129 ceph-mon[30979]: pgmap v407: 129 pgs: 129 active+clean; 956 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 65 op/s 2024-09-05T01:17:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:12 smithi194 ceph-mon[34941]: pgmap v407: 129 pgs: 129 active+clean; 956 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 65 op/s 2024-09-05T01:17:13.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:12 smithi183 ceph-mon[34589]: pgmap v407: 129 pgs: 129 active+clean; 956 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 65 op/s 2024-09-05T01:17:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 10.0 s, 1240.2 tps, lat 80.281 ms stddev 23.688 2024-09-05T01:17:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:14 smithi129 ceph-mon[30979]: pgmap v408: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 298 op/s 2024-09-05T01:17:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:14 smithi194 ceph-mon[34941]: pgmap v408: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 298 op/s 2024-09-05T01:17:15.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:14 smithi183 ceph-mon[34589]: pgmap v408: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 34 MiB/s wr, 298 op/s 2024-09-05T01:17:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:16 smithi129 ceph-mon[30979]: pgmap v409: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 18 MiB/s wr, 330 op/s 2024-09-05T01:17:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:16 smithi194 ceph-mon[34941]: pgmap v409: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 18 MiB/s wr, 330 op/s 2024-09-05T01:17:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:16 smithi183 ceph-mon[34589]: pgmap v409: 129 pgs: 129 active+clean; 928 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 18 MiB/s wr, 330 op/s 2024-09-05T01:17:19.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:18 smithi129 ceph-mon[30979]: pgmap v410: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 502 op/s 2024-09-05T01:17:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:18 smithi194 ceph-mon[34941]: pgmap v410: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 502 op/s 2024-09-05T01:17:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:18 smithi183 ceph-mon[34589]: pgmap v410: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 16 MiB/s wr, 502 op/s 2024-09-05T01:17:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 15.0 s, 1146.0 tps, lat 87.178 ms stddev 22.744 2024-09-05T01:17:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:20 smithi129 ceph-mon[30979]: pgmap v411: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 842 op/s 2024-09-05T01:17:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:20 smithi194 ceph-mon[34941]: pgmap v411: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 842 op/s 2024-09-05T01:17:21.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:20 smithi183 ceph-mon[34589]: pgmap v411: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 12 MiB/s wr, 842 op/s 2024-09-05T01:17:22.444 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.2 2024-09-05T01:17:22.445 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:17:22.893 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:17:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:22 smithi129 ceph-mon[30979]: pgmap v412: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.5 MiB/s wr, 875 op/s 2024-09-05T01:17:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:22 smithi194 ceph-mon[34941]: pgmap v412: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.5 MiB/s wr, 875 op/s 2024-09-05T01:17:23.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:22 smithi183 ceph-mon[34589]: pgmap v412: 129 pgs: 129 active+clean; 928 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.5 MiB/s wr, 875 op/s 2024-09-05T01:17:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.108+0000 7f4324817640 1 -- 172.21.15.129:0/2520249879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43200ff8e0 msgr2=0x7f43200ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.108+0000 7f4324817640 1 --2- 172.21.15.129:0/2520249879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43200ff8e0 0x7f43200ffce0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4310009860 tx=0x7f431002a340 comp rx=0 tx=0).stop 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 -- 172.21.15.129:0/2520249879 shutdown_connections 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 --2- 172.21.15.129:0/2520249879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43201077d0 0x7f4320109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 --2- 172.21.15.129:0/2520249879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4320100220 0x7f4320107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 --2- 172.21.15.129:0/2520249879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f43200ff8e0 0x7f43200ffce0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 -- 172.21.15.129:0/2520249879 >> 172.21.15.129:0/2520249879 conn(0x7f43200fb500 msgr2=0x7f43200fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 -- 172.21.15.129:0/2520249879 shutdown_connections 2024-09-05T01:17:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.109+0000 7f4324817640 1 -- 172.21.15.129:0/2520249879 wait complete. 2024-09-05T01:17:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 Processor -- start 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 -- start start 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 0x7f4320102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43201077d0 0x7f4320101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d750 con 0x7f43200ff8e0 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d5d0 con 0x7f43201077d0 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f4324817640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d8d0 con 0x7f4320100220 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f431e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 0x7f4320102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f431ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43201077d0 0x7f4320101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.110+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57882/0 (socket says 172.21.15.129:57882) 2024-09-05T01:17:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 -- 172.21.15.129:0/1073116213 learned_addr learned my addr 172.21.15.129:0/1073116213 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 -- 172.21.15.129:0/1073116213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43201077d0 msgr2=0x7f4320101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43201077d0 0x7f4320101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 -- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 msgr2=0x7f4320102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 0x7f4320102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 -- 172.21.15.129:0/1073116213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4320101c50 con 0x7f4320100220 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431e59c640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 0x7f4320102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f431400e9a0 tx=0x7f431400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.111+0000 7f430f7fe640 1 -- 172.21.15.129:0/1073116213 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431400cd80 con 0x7f4320100220 2024-09-05T01:17:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.112+0000 7f430f7fe640 1 -- 172.21.15.129:0/1073116213 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43140035f0 con 0x7f4320100220 2024-09-05T01:17:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.112+0000 7f430f7fe640 1 -- 172.21.15.129:0/1073116213 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4314010690 con 0x7f4320100220 2024-09-05T01:17:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.112+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4320101f40 con 0x7f4320100220 2024-09-05T01:17:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.112+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43201a67d0 con 0x7f4320100220 2024-09-05T01:17:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.115+0000 7f4327689740 1 Processor -- start 2024-09-05T01:17:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 -- start start 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f430f7fe640 1 -- 172.21.15.129:0/1073116213 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f43140037a0 con 0x7f4320100220 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a0196dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 0x563a01970830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563a018cc2f0 con 0x563a019689a0 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a01874270 con 0x563a019692e0 2024-09-05T01:17:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f4327689740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a018c82b0 con 0x563a0196e370 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f431ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f431ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40638/0 (socket says 172.21.15.129:40638) 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.116+0000 7f431ed9d640 1 -- 172.21.15.129:0/1765483325 learned_addr learned my addr 172.21.15.129:0/1765483325 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 0x563a01970830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431ed9d640 1 -- 172.21.15.129:0/1765483325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 msgr2=0x563a01970830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431ed9d640 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 0x563a01970830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431ed9d640 1 -- 172.21.15.129:0/1765483325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 msgr2=0x563a0196dda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431ed9d640 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a0196dda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431ed9d640 1 -- 172.21.15.129:0/1765483325 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a01873340 con 0x563a019689a0 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.117+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 0x563a01970830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.118+0000 7f431ed9d640 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f431800a430 tx=0x7f4318009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baabae8a4ff42034 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.118+0000 7f430f7fe640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 0x7f43080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.118+0000 7f430f7fe640 1 -- 172.21.15.129:0/1073116213 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f43140cf6c0 con 0x7f4320100220 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.118+0000 7f431e59c640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 0x7f43080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.118+0000 7f430cff9640 1 -- 172.21.15.129:0/1765483325 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318013790 con 0x563a019689a0 2024-09-05T01:17:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.119+0000 7f430cff9640 1 -- 172.21.15.129:0/1765483325 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4318019070 con 0x563a019689a0 2024-09-05T01:17:24.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.119+0000 7f430cff9640 1 -- 172.21.15.129:0/1765483325 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4318014ce0 con 0x563a019689a0 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 -- 172.21.15.129:0/1765483325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 msgr2=0x563a01968da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f431800a430 tx=0x7f4318009650 comp rx=0 tx=0).stop 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f431e59c640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 0x7f43080a51c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4310000c00 tx=0x7f4310030450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 -- 172.21.15.129:0/1765483325 shutdown_connections 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a0196e370 0x563a01970830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a0196dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 --2- 172.21.15.129:0/1765483325 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a01968da0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 -- 172.21.15.129:0/1765483325 >> 172.21.15.129:0/1765483325 conn(0x563a01964010 msgr2=0x563a01964410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.120+0000 7f4327689740 1 -- 172.21.15.129:0/1765483325 shutdown_connections 2024-09-05T01:17:24.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.121+0000 7f4327689740 1 -- 172.21.15.129:0/1765483325 wait complete. 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.121+0000 7f4327689740 1 -- start start 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.121+0000 7f4327689740 1 Processor -- start 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f4327689740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019689a0 0x563a01972c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f4327689740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f4327689740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a0197a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:44806/0 (socket says 172.21.15.129:44806) 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f431dd9b640 1 -- 172.21.15.129:0/1800787337 learned_addr learned my addr 172.21.15.129:0/1800787337 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f431e59c640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019689a0 0x563a01972c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f431ed9d640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a0197a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563a019713e0 con 0x563a0196e370 2024-09-05T01:17:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.122+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a01855290 con 0x563a019692e0 2024-09-05T01:17:24.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563a01971560 con 0x563a019689a0 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f4327689740 10 client.0 _ll_register_callbacks cb 0x563a01886140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 -- 172.21.15.129:0/1800787337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019689a0 msgr2=0x563a01972c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019689a0 0x563a01972c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 -- 172.21.15.129:0/1800787337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 msgr2=0x563a0197a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a0197a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a015df210 con 0x563a019692e0 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.123+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43140a41f0 tx=0x7f431409b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8da891bc357c0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.124+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4318019070 con 0x563a019692e0 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.124+0000 7f4305ffb640 10 client.25115 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.124+0000 7f4305ffb640 1 -- 172.21.15.129:0/1800787337 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431409bab0 con 0x563a019692e0 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.124+0000 7f4305ffb640 1 -- 172.21.15.129:0/1800787337 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f43140a4290 con 0x563a019692e0 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.124+0000 7f4305ffb640 1 -- 172.21.15.129:0/1800787337 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f431409fe60 con 0x563a019692e0 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.125+0000 7f4327689740 10 client.25115 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:24.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.125+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=27}) -- 0x563a016ab620 con 0x563a019692e0 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4305ffb640 1 -- 172.21.15.129:0/1800787337 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f43140ba290 con 0x563a019692e0 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 10 client.25115 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 10 client.25115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 20 client.25115 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a0197ead0 0x563a01980f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 4 client.25115 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4318019070 con 0x563a0197ead0 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.126+0000 7f431e59c640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a0197ead0 0x563a01980f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.127+0000 7f4305ffb640 10 client.25115 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.127+0000 7f431e59c640 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a0197ead0 0x563a01980f00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=232397325ea8f1cb server_cookie=6b1eb418ad0a4eb4 in_seq=0 out_seq=0 2024-09-05T01:17:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4305ffb640 1 -- 172.21.15.129:0/1800787337 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4318019070 con 0x563a0197ead0 2024-09-05T01:17:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4305ffb640 10 client.25115 handle_command_reply: tid=0 2024-09-05T01:17:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 client.25115 shutdown 2024-09-05T01:17:24.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 msgr2=0x563a019731b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f43140a41f0 tx=0x7f431409b410 comp rx=0 tx=0).stop 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 shutdown_connections 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a0197ead0 0x563a01980f00 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a0197a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019692e0 0x563a019731b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 --2- 172.21.15.129:0/1800787337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019689a0 0x563a01972c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 >> 172.21.15.129:0/1800787337 conn(0x563a01964010 msgr2=0x563a01965820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4305ffb640 0 client.25115 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 shutdown_connections 2024-09-05T01:17:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.128+0000 7f4327689740 1 -- 172.21.15.129:0/1800787337 wait complete. 2024-09-05T01:17:24.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.129+0000 7f4327689740 20 client.25115 trim_cache size 0 max 16384 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 Processor -- start 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 -- start start 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a019643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a0196e370 0x563a01964930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43201b3890 con 0x563a019689a0 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43201b1160 con 0x563a0196e370 2024-09-05T01:17:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.157+0000 7f4327689740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43201aefb0 con 0x563a019692e0 2024-09-05T01:17:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57894/0 (socket says 172.21.15.129:57894) 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 -- 172.21.15.129:0/2626405917 learned_addr learned my addr 172.21.15.129:0/2626405917 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431ed9d640 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a0196e370 0x563a01964930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431e59c640 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a019643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 -- 172.21.15.129:0/2626405917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a0196e370 msgr2=0x563a01964930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a0196e370 0x563a01964930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 -- 172.21.15.129:0/2626405917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 msgr2=0x563a019643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a019643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 -- 172.21.15.129:0/2626405917 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a01963e30 con 0x563a019692e0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.158+0000 7f431dd9b640 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43140096c0 tx=0x7f4314019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec3c7425392782f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.159+0000 7f4307fff640 1 -- 172.21.15.129:0/2626405917 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431409bab0 con 0x563a019692e0 2024-09-05T01:17:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.159+0000 7f4307fff640 1 -- 172.21.15.129:0/2626405917 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43140a4720 con 0x563a019692e0 2024-09-05T01:17:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.159+0000 7f4307fff640 1 -- 172.21.15.129:0/2626405917 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43140d1b80 con 0x563a019692e0 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 -- 172.21.15.129:0/2626405917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 msgr2=0x563a0197a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43140096c0 tx=0x7f4314019000 comp rx=0 tx=0).stop 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 -- 172.21.15.129:0/2626405917 shutdown_connections 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a0196e370 0x563a01964930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a0197a1b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 --2- 172.21.15.129:0/2626405917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a019689a0 0x563a019643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.160+0000 7f4327689740 1 -- 172.21.15.129:0/2626405917 >> 172.21.15.129:0/2626405917 conn(0x563a01964010 msgr2=0x563a01972c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.161+0000 7f4327689740 1 -- 172.21.15.129:0/2626405917 shutdown_connections 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.161+0000 7f4327689740 1 -- 172.21.15.129:0/2626405917 wait complete. 2024-09-05T01:17:24.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.161+0000 7f4327689740 1 -- start start 2024-09-05T01:17:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f4327689740 1 Processor -- start 2024-09-05T01:17:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f4327689740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f4327689740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a019778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f4327689740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a019788a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f431e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f431e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:44822/0 (socket says 172.21.15.129:44822) 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.162+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a019778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a019778e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57898/0 (socket says 172.21.15.129:57898) 2024-09-05T01:17:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4327689740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f43201af620 con 0x563a0196e370 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4327689740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f43201b0a90 con 0x563a019689a0 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4327689740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f43201afcf0 con 0x563a019692e0 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4327689740 10 client.0 _ll_register_callbacks cb 0x7f4314004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 -- 172.21.15.129:0/1804937344 learned_addr learned my addr 172.21.15.129:0/1804937344 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431ed9d640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a019788a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 -- 172.21.15.129:0/1804937344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 msgr2=0x563a019778e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a019778e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 -- 172.21.15.129:0/1804937344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 msgr2=0x563a019788a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a019788a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 -- 172.21.15.129:0/1804937344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a01978eb0 con 0x563a019689a0 2024-09-05T01:17:24.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f431e59c640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4310009860 tx=0x7f4310005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7acedfe7dc623747 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.163+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310059060 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43140a4720 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f4310005450 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4310060050 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 10 client.25121 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563a0197a290 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f43140a4720 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f4310003710 con 0x563a019689a0 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 10 client.25121 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 10 client.25121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:24.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 20 client.25121 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.164+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a019e9c70 0x563a019ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.165+0000 7f4327689740 4 client.25121 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.165+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a018d9a20 con 0x563a019e9c70 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.165+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a019e9c70 0x563a019ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.165+0000 7f431dd9b640 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a019e9c70 0x563a019ec090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841f7874cbbec23e server_cookie=5e468c815a4c493c in_seq=0 out_seq=0 2024-09-05T01:17:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.165+0000 7f4305ffb640 10 client.25121 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.206+0000 7f4305ffb640 1 -- 172.21.15.129:0/1804937344 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a018d9a20 con 0x563a019e9c70 2024-09-05T01:17:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.206+0000 7f4305ffb640 10 client.25121 handle_command_reply: tid=0 2024-09-05T01:17:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.206+0000 7f4327689740 1 client.25121 shutdown 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 msgr2=0x563a019773a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4310009860 tx=0x7f4310005f90 comp rx=0 tx=0).stop 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 shutdown_connections 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563a019e9c70 0x563a019ec090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563a0196e370 0x563a019788a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4305ffb640 0 client.25121 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563a019692e0 0x563a019778e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 --2- 172.21.15.129:0/1804937344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a019689a0 0x563a019773a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 >> 172.21.15.129:0/1804937344 conn(0x563a01964010 msgr2=0x563a019629a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 shutdown_connections 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.207+0000 7f4327689740 1 -- 172.21.15.129:0/1804937344 wait complete. 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.208+0000 7f4327689740 20 client.25121 trim_cache size 0 max 16384 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:17:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:17:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 msgr2=0x7f43080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 0x7f43080a51c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4310000c00 tx=0x7f4310030450 comp rx=0 tx=0).stop 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 msgr2=0x7f43201011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f431400e9a0 tx=0x7f431400ee60 comp rx=0 tx=0).stop 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 shutdown_connections 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f43080a2d00 0x7f43080a51c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f43201077d0 0x7f4320101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4320100220 0x7f43201011d0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f431400e9a0 tx=0x7f431400ee60 comp rx=0 tx=0).stop 2024-09-05T01:17:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 --2- 172.21.15.129:0/1073116213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f43200ff8e0 0x7f4320102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.210+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 >> 172.21.15.129:0/1073116213 conn(0x7f43200fb500 msgr2=0x7f4320103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.211+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 shutdown_connections 2024-09-05T01:17:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:24.211+0000 7f4324817640 1 -- 172.21.15.129:0/1073116213 wait complete. 2024-09-05T01:17:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 20.0 s, 1034.2 tps, lat 96.546 ms stddev 30.447 2024-09-05T01:17:25.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:24 smithi129 ceph-mon[30979]: pgmap v413: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-09-05T01:17:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:24 smithi194 ceph-mon[34941]: pgmap v413: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-09-05T01:17:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:24 smithi183 ceph-mon[34589]: pgmap v413: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 1.31k op/s 2024-09-05T01:17:25.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 -- 172.21.15.129:0/1891030102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 msgr2=0x7f2400101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:25.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 --2- 172.21.15.129:0/1891030102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f2400101200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009890 tx=0x7f23f002a400 comp rx=0 tx=0).stop 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 -- 172.21.15.129:0/1891030102 shutdown_connections 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 --2- 172.21.15.129:0/1891030102 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2400102ef0 0x7f240010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 --2- 172.21.15.129:0/1891030102 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2400102570 0x7f24001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 --2- 172.21.15.129:0/1891030102 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f2400101200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 -- 172.21.15.129:0/1891030102 >> 172.21.15.129:0/1891030102 conn(0x7f24000fc7d0 msgr2=0x7f24000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 -- 172.21.15.129:0/1891030102 shutdown_connections 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.778+0000 7f2406905640 1 -- 172.21.15.129:0/1891030102 wait complete. 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.779+0000 7f2406905640 1 Processor -- start 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.779+0000 7f2406905640 1 -- start start 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2400102570 0x7f240019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:25.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2400102ef0 0x7f240019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2400110e80 con 0x7f2400102ef0 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2400110d00 con 0x7f2400102570 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f2406905640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2400111000 con 0x7f2400100e20 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57922/0 (socket says 172.21.15.129:57922) 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 -- 172.21.15.129:0/2080019635 learned_addr learned my addr 172.21.15.129:0/2080019635 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 -- 172.21.15.129:0/2080019635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2400102570 msgr2=0x7f240019ab80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2400102570 0x7f240019ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 -- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2400102ef0 msgr2=0x7f240019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2400102ef0 0x7f240019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.780+0000 7f23fffff640 1 -- 172.21.15.129:0/2080019635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240019f5f0 con 0x7f2400100e20 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.781+0000 7f23fffff640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f0000c00 tx=0x7f23f0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:25.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.781+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0037050 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.781+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23f0054070 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.782+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0030af0 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.782+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240019f880 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.782+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240019fc20 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.784+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2400101200 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.785+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f23f002bd10 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.786+0000 7f23fd7fa640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 0x7f23e80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.786+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f23f01008d0 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.787+0000 7f23ff7fe640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 0x7f23e80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.787+0000 7f23ff7fe640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 0x7f23e80a51d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f24000fec10 tx=0x7f23f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:24.791+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23f00cd420 con 0x7f2400100e20 2024-09-05T01:17:25.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.153+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2400104d60 con 0x7f2400100e20 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.153+0000 7f23fd7fa640 1 -- 172.21.15.129:0/2080019635 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f23f00ccd20 con 0x7f2400100e20 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"btime":"2024-09-05T01:15:41:859752+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:41.039933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 27 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 msgr2=0x7f23e80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:25.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 0x7f23e80a51d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f24000fec10 tx=0x7f23f4005f90 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 msgr2=0x7f240019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23f0000c00 tx=0x7f23f0004270 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 shutdown_connections 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f23e80a2d10 0x7f23e80a51d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2400102ef0 0x7f240019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2400102570 0x7f240019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.158+0000 7f2406905640 1 --2- 172.21.15.129:0/2080019635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2400100e20 0x7f240019a640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.159+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 >> 172.21.15.129:0/2080019635 conn(0x7f24000fc7d0 msgr2=0x7f240006a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.159+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 shutdown_connections 2024-09-05T01:17:25.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:25.159+0000 7f2406905640 1 -- 172.21.15.129:0/2080019635 wait complete. 2024-09-05T01:17:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2080019635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2080019635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:26.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2080019635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:26 smithi129 ceph-mon[30979]: pgmap v414: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-05T01:17:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:26 smithi194 ceph-mon[34941]: pgmap v414: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-05T01:17:27.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:26 smithi183 ceph-mon[34589]: pgmap v414: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 MiB/s wr, 1.16k op/s 2024-09-05T01:17:29.213 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:28 smithi183 ceph-mon[34589]: pgmap v415: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-05T01:17:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:28 smithi129 ceph-mon[30979]: pgmap v415: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-05T01:17:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:28 smithi194 ceph-mon[34941]: pgmap v415: 129 pgs: 129 active+clean; 928 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 11 MiB/s wr, 1.25k op/s 2024-09-05T01:17:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 25.0 s, 891.4 tps, lat 112.540 ms stddev 59.684 2024-09-05T01:17:30.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:30 smithi183 ceph-mon[34589]: pgmap v416: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-09-05T01:17:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:30 smithi129 ceph-mon[30979]: pgmap v416: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-09-05T01:17:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:30 smithi194 ceph-mon[34941]: pgmap v416: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 12 MiB/s wr, 1.37k op/s 2024-09-05T01:17:33.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:32 smithi183 ceph-mon[34589]: pgmap v417: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:17:33.201 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:32 smithi194 ceph-mon[34941]: pgmap v417: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:17:33.205 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:32 smithi129 ceph-mon[30979]: pgmap v417: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:17:33.259 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0' 2024-09-05T01:17:33.259 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 144.2 --await-for 45.0 2024-09-05T01:17:33.630 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.182+0000 7fe0c7fff640 1 -- 172.21.15.129:0/3026635055 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0031830 con 0x7fe0d0102570 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 -- 172.21.15.129:0/3026635055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 msgr2=0x7fe0d01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 --2- 172.21.15.129:0/3026635055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01029b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009950 tx=0x7fe0c002a4d0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 -- 172.21.15.129:0/3026635055 shutdown_connections 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 --2- 172.21.15.129:0/3026635055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0102ef0 0x7fe0d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.185 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 --2- 172.21.15.129:0/3026635055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01029b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 --2- 172.21.15.129:0/3026635055 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0100e20 0x7fe0d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 -- 172.21.15.129:0/3026635055 >> 172.21.15.129:0/3026635055 conn(0x7fe0d00fc7d0 msgr2=0x7fe0d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.186 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 -- 172.21.15.129:0/3026635055 shutdown_connections 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.183+0000 7fe0d72db640 1 -- 172.21.15.129:0/3026635055 wait complete. 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.184+0000 7fe0d72db640 1 Processor -- start 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.184+0000 7fe0d72db640 1 -- start start 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010fd30 con 0x7fe0d0102570 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010fbb0 con 0x7fe0d0100e20 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d72db640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d010feb0 con 0x7fe0d0102ef0 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.187 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:52882/0 (socket says 172.21.15.129:52882) 2024-09-05T01:17:34.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:60552/0 (socket says 172.21.15.129:60552) 2024-09-05T01:17:34.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 -- 172.21.15.129:0/786892091 learned_addr learned my addr 172.21.15.129:0/786892091 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:34.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49480/0 (socket says 172.21.15.129:49480) 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 -- 172.21.15.129:0/786892091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 msgr2=0x7fe0d01a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 -- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 msgr2=0x7fe0d01a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5878640 1 -- 172.21.15.129:0/786892091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d019efa0 con 0x7fe0d0102ef0 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.185+0000 7fe0d5077640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0d4876640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0d5878640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800da90 tx=0x7fe0c800df50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0c67fc640 1 -- 172.21.15.129:0/786892091 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c801b020 con 0x7fe0d0102ef0 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0c67fc640 1 -- 172.21.15.129:0/786892091 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c800bc30 con 0x7fe0d0102ef0 2024-09-05T01:17:34.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0c67fc640 1 -- 172.21.15.129:0/786892091 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005970 con 0x7fe0d0102ef0 2024-09-05T01:17:34.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d019f290 con 0x7fe0d0102ef0 2024-09-05T01:17:34.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.186+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d01ab580 con 0x7fe0d0102ef0 2024-09-05T01:17:34.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.189+0000 7fe0c67fc640 1 -- 172.21.15.129:0/786892091 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe0c8004b20 con 0x7fe0d0102ef0 2024-09-05T01:17:34.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.190+0000 7fe0d96b4740 1 Processor -- start 2024-09-05T01:17:34.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.190+0000 7fe0d96b4740 1 -- start start 2024-09-05T01:17:34.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a8231b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0c67fc640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 0x7fe0bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a8231b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a8231b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49488/0 (socket says 172.21.15.129:49488) 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0c67fc640 1 -- 172.21.15.129:0/786892091 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe0c80cf2a0 con 0x7fe0d0102ef0 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d4876640 1 -- 172.21.15.129:0/2805477839 learned_addr learned my addr 172.21.15.129:0/2805477839 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5590a81972a0 con 0x5590a8231760 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5590a813eb10 con 0x5590a823c940 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590a8193260 con 0x5590a8232080 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d5077640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 0x7fe0bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.191+0000 7fe0d4876640 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 -- 172.21.15.129:0/2805477839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 msgr2=0x5590a823c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 -- 172.21.15.129:0/2805477839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 msgr2=0x5590a8231b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a8231b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 -- 172.21.15.129:0/2805477839 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a7f08b90 con 0x5590a823c940 2024-09-05T01:17:34.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823ed30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80cc930 tx=0x7fe0c8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2efa15b78d9cf65 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0bb7fe640 1 -- 172.21.15.129:0/2805477839 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c809be20 con 0x5590a823c940 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0bb7fe640 1 -- 172.21.15.129:0/2805477839 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c800bde0 con 0x5590a823c940 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.192+0000 7fe0bb7fe640 1 -- 172.21.15.129:0/2805477839 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80997c0 con 0x5590a823c940 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.193+0000 7fe0d5077640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 0x7fe0bc0a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00f640 tx=0x7fe0cc011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.193+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 msgr2=0x5590a823ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.193+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823ed30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe0c80cc930 tx=0x7fe0c8004a40 comp rx=0 tx=0).stop 2024-09-05T01:17:34.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 shutdown_connections 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823ed30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2805477839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a8231b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 >> 172.21.15.129:0/2805477839 conn(0x5590a822dcf0 msgr2=0x5590a822e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 shutdown_connections 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.194+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2805477839 wait complete. 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.195+0000 7fe0d96b4740 1 -- start start 2024-09-05T01:17:34.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.195+0000 7fe0d96b4740 1 Processor -- start 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a82398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a8239df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5590a823f510 con 0x5590a8231760 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5590a811f6e0 con 0x5590a823c940 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5590a823f690 con 0x5590a8232080 2024-09-05T01:17:34.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d96b4740 10 client.0 _ll_register_callbacks cb 0x5590a8143a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a8239df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d5077640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a82398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a8239df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:52896/0 (socket says 172.21.15.129:52896) 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d5878640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.196+0000 7fe0d4876640 1 -- 172.21.15.129:0/2537906030 learned_addr learned my addr 172.21.15.129:0/2537906030 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 -- 172.21.15.129:0/2537906030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 msgr2=0x5590a8239df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a8239df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 -- 172.21.15.129:0/2537906030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 msgr2=0x5590a82398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a82398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.199 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 -- 172.21.15.129:0/2537906030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a8248ad0 con 0x5590a823c940 2024-09-05T01:17:34.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823a330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0c809b8e0 tx=0x7fe0c8004e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6116d2983e5e53c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.197+0000 7fe0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80b9060 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe0c8099a20 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c800bde0 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0c809b720 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0d96b4740 10 client.25136 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590a824b460 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.198+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c800bde0 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe0c8016070 con 0x5590a823c940 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 10 client.25136 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 10 client.25136 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:17:34.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 20 client.25136 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:34.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a824eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 4 client.25136 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:34.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a81a49d0 con 0x5590a824c7b0 2024-09-05T01:17:34.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.199+0000 7fe0d5077640 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a824eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.200+0000 7fe0b8ff9640 10 client.25136 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.200+0000 7fe0d5077640 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a824eba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53cdd51d0d8ecdf1 server_cookie=53aaf4152a52a97d in_seq=0 out_seq=0 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.201+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2537906030 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590a81a49d0 con 0x5590a824c7b0 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.201+0000 7fe0b8ff9640 10 client.25136 handle_command_reply: tid=0 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.201+0000 7fe0d96b4740 1 client.25136 shutdown 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 msgr2=0x5590a823a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823a330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe0c809b8e0 tx=0x7fe0c8004e10 comp rx=0 tx=0).stop 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 shutdown_connections 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a824eba0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a823c940 0x5590a823a330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a8239df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2537906030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8231760 0x5590a82398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0b8ff9640 0 client.25136 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 >> 172.21.15.129:0/2537906030 conn(0x5590a822dcf0 msgr2=0x5590a8233a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.202+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 shutdown_connections 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.203+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2537906030 wait complete. 2024-09-05T01:17:34.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.203+0000 7fe0d96b4740 20 client.25136 trim_cache size 0 max 16384 2024-09-05T01:17:34.231 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.230+0000 7fe0d96b4740 1 Processor -- start 2024-09-05T01:17:34.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 -- start start 2024-09-05T01:17:34.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8238bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a823c940 0x5590a822cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b7cf0 con 0x5590a823c940 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b55c0 con 0x5590a8231760 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d96b4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b3410 con 0x5590a8232080 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d4876640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:52902/0 (socket says 172.21.15.129:52902) 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d4876640 1 -- 172.21.15.129:0/61517388 learned_addr learned my addr 172.21.15.129:0/61517388 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d5077640 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8238bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.231+0000 7fe0d5878640 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a823c940 0x5590a822cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 -- 172.21.15.129:0/61517388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 msgr2=0x5590a8238bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8238bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 -- 172.21.15.129:0/61517388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a823c940 msgr2=0x5590a822cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a823c940 0x5590a822cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 -- 172.21.15.129:0/61517388 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a8229260 con 0x5590a8232080 2024-09-05T01:17:34.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0d4876640 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009950 tx=0x7fe0c0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eab77811bb8310c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0baffd640 1 -- 172.21.15.129:0/61517388 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c004e070 con 0x5590a8232080 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0baffd640 1 -- 172.21.15.129:0/61517388 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c0004c30 con 0x5590a8232080 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.232+0000 7fe0baffd640 1 -- 172.21.15.129:0/61517388 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0035040 con 0x5590a8232080 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.233+0000 7fe0d96b4740 1 -- 172.21.15.129:0/61517388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 msgr2=0x5590a823b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.233+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009950 tx=0x7fe0c0005fd0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 -- 172.21.15.129:0/61517388 shutdown_connections 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a823c940 0x5590a822cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a8232080 0x5590a823b780 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/61517388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8238bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 -- 172.21.15.129:0/61517388 >> 172.21.15.129:0/61517388 conn(0x5590a822dcf0 msgr2=0x5590a822c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 -- 172.21.15.129:0/61517388 shutdown_connections 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 -- 172.21.15.129:0/61517388 wait complete. 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.234+0000 7fe0d96b4740 1 -- start start 2024-09-05T01:17:34.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.235+0000 7fe0d96b4740 1 Processor -- start 2024-09-05T01:17:34.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.235+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.235+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 0x5590a8252280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d96b4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a823c940 0x5590a8250130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:60580/0 (socket says 172.21.15.129:60580) 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 -- 172.21.15.129:0/2152444895 learned_addr learned my addr 172.21.15.129:0/2152444895 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b3a80 con 0x5590a8232080 2024-09-05T01:17:34.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b4ef0 con 0x5590a8231760 2024-09-05T01:17:34.239 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d01b4150 con 0x5590a823c940 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d96b4740 10 client.0 _ll_register_callbacks cb 0x7fe0c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5878640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a823c940 0x5590a8250130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d4876640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 0x5590a8252280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 -- 172.21.15.129:0/2152444895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a823c940 msgr2=0x5590a8250130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a823c940 0x5590a8250130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 -- 172.21.15.129:0/2152444895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 msgr2=0x5590a8252280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 0x5590a8252280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d5077640 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a8250960 con 0x5590a8231760 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.236+0000 7fe0d4876640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 0x5590a8252280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0d5077640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc01a650 tx=0x7fe0cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e859fafb3574be06 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc034030 con 0x5590a8231760 2024-09-05T01:17:34.240 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe0cc02d040 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0c0004c30 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0cc03e050 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0d96b4740 10 client.25142 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.237+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590a82b7be0 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0c0004c30 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe0cc028410 con 0x5590a8231760 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 10 client.25142 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 10 client.25142 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 20 client.25142 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a82ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 4 client.25142 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}] 2024-09-05T01:17:34.241 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.2, "await_for": 45.0}) -- 0x5590a819ed20 con 0x5590a824c7b0 2024-09-05T01:17:34.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.238+0000 7fe0d4876640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a82ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:34.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.239+0000 7fe0b8ff9640 10 client.25142 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:34.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.239+0000 7fe0d4876640 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a82ba3f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38be416142748830 server_cookie=2391a1f44df95450 in_seq=0 out_seq=0 2024-09-05T01:17:34.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:34 smithi183 ceph-mon[34589]: pgmap v418: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-05T01:17:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:34 smithi129 ceph-mon[30979]: pgmap v418: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-05T01:17:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:34 smithi194 ceph-mon[34941]: pgmap v418: 129 pgs: 129 active+clean; 928 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 1.31k op/s 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0b8ff9640 1 -- 172.21.15.129:0/2152444895 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5590a819ed20 con 0x5590a824c7b0 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0b8ff9640 10 client.25142 handle_command_reply: tid=0 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 client.25142 shutdown 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 msgr2=0x5590a8251d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc01a650 tx=0x7fe0cc0046c0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 shutdown_connections 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5590a824c7b0 0x5590a82ba3f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5590a823c940 0x5590a8250130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5590a8232080 0x5590a8252280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0b8ff9640 0 client.25142 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:34.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 --2- 172.21.15.129:0/2152444895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590a8231760 0x5590a8251d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.512+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 >> 172.21.15.129:0/2152444895 conn(0x5590a822dcf0 msgr2=0x5590a8248150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 shutdown_connections 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d96b4740 1 -- 172.21.15.129:0/2152444895 wait complete. 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d96b4740 20 client.25142 trim_cache size 0 max 16384 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 27, 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 6, 2024-09-05T01:17:34.516 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "9ec3ddd2": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 6, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 144.2, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:17:34.517 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:34.518 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 msgr2=0x7fe0bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 0x7fe0bc0a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00f640 tx=0x7fe0cc011040 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 msgr2=0x7fe0d019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800da90 tx=0x7fe0c800df50 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 shutdown_connections 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0bc0a2c00 0x7fe0bc0a50c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0d0102ef0 0x7fe0d019e870 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0d0102570 0x7fe0d01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 --2- 172.21.15.129:0/786892091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0d0100e20 0x7fe0d01a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.513+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 >> 172.21.15.129:0/786892091 conn(0x7fe0d00fc7d0 msgr2=0x7fe0d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.514+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 shutdown_connections 2024-09-05T01:17:34.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:34.514+0000 7fe0d72db640 1 -- 172.21.15.129:0/786892091 wait complete. 2024-09-05T01:17:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 30.0 s, 693.5 tps, lat 117.318 ms stddev 120.998 2024-09-05T01:17:35.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9ec3ddd2', quiesce took 12.8 seconds. Will release after: 11.399999999999999 2024-09-05T01:17:36.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:36 smithi129 ceph-mon[30979]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.00k op/s 2024-09-05T01:17:37.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:36 smithi183 ceph-mon[34589]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.00k op/s 2024-09-05T01:17:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:36 smithi194 ceph-mon[34941]: pgmap v419: 129 pgs: 129 active+clean; 932 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.8 MiB/s wr, 1.00k op/s 2024-09-05T01:17:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- 172.21.15.129:0/4219020779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c100a50 msgr2=0x7fcd5c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/4219020779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c100a50 0x7fcd5c100e50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009900 tx=0x7fcd5002a410 comp rx=0 tx=0).stop 2024-09-05T01:17:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- 172.21.15.129:0/4219020779 shutdown_connections 2024-09-05T01:17:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/4219020779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 0x7fcd5c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/4219020779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c101c40 0x7fcd5c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/4219020779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c100a50 0x7fcd5c100e50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- 172.21.15.129:0/4219020779 >> 172.21.15.129:0/4219020779 conn(0x7fcd5c0fc1b0 msgr2=0x7fcd5c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- 172.21.15.129:0/4219020779 shutdown_connections 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- 172.21.15.129:0/4219020779 wait complete. 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 Processor -- start 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 -- start start 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 0x7fcd5c06ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.315+0000 7fcd62ca5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd62ca5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 0x7fcd5c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd62ca5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10c580 con 0x7fcd5c101c40 2024-09-05T01:17:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd62ca5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10c400 con 0x7fcd5c100a50 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd62ca5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c10c700 con 0x7fcd5c102600 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49560/0 (socket says 172.21.15.129:49560) 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 -- 172.21.15.129:0/2755216083 learned_addr learned my addr 172.21.15.129:0/2755216083 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd61242640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 0x7fcd5c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd60a41640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 0x7fcd5c06ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 -- 172.21.15.129:0/2755216083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 msgr2=0x7fcd5c19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 0x7fcd5c19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 -- 172.21.15.129:0/2755216083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 msgr2=0x7fcd5c06ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 0x7fcd5c06ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 -- 172.21.15.129:0/2755216083 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c19f980 con 0x7fcd5c101c40 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd60a41640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 0x7fcd5c06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9f0 tx=0x7fcd4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd59ffb640 1 -- 172.21.15.129:0/2755216083 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4800cd80 con 0x7fcd5c101c40 2024-09-05T01:17:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd59ffb640 1 -- 172.21.15.129:0/2755216083 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd480108e0 con 0x7fcd5c101c40 2024-09-05T01:17:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19fc10 con 0x7fcd5c101c40 2024-09-05T01:17:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.316+0000 7fcd59ffb640 1 -- 172.21.15.129:0/2755216083 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48018920 con 0x7fcd5c101c40 2024-09-05T01:17:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.317+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a01b0 con 0x7fcd5c101c40 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd59ffb640 1 -- 172.21.15.129:0/2755216083 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcd48010a90 con 0x7fcd5c101c40 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 Processor -- start 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 -- start start 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c033787da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c033788370 0x55c03378a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c0336e62f0 con 0x55c0337829a0 2024-09-05T01:17:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c03368e270 con 0x55c0337832e0 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd65a46740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c0336e22b0 con 0x55c033788370 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.319+0000 7fcd59ffb640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 0x7fcd4c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 0x7fcd4c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49570/0 (socket says 172.21.15.129:49570) 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd60a41640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c033787da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 -- 172.21.15.129:0/152331106 learned_addr learned my addr 172.21.15.129:0/152331106 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd59ffb640 1 -- 172.21.15.129:0/2755216083 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fcd480cf730 con 0x7fcd5c101c40 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c033788370 0x55c03378a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 -- 172.21.15.129:0/152331106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c033788370 msgr2=0x55c03378a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c033788370 0x55c03378a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 -- 172.21.15.129:0/152331106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 msgr2=0x55c033787da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c033787da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 -- 172.21.15.129:0/152331106 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03368d340 con 0x55c0337829a0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd54010e50 tx=0x7fcd5400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797034138516ccf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd61242640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 0x7fcd4c0a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400fbd0 tx=0x7fcd5401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.320+0000 7fcd46ffd640 1 -- 172.21.15.129:0/152331106 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54019040 con 0x55c0337829a0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd46ffd640 1 -- 172.21.15.129:0/152331106 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd54009e00 con 0x55c0337829a0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd46ffd640 1 -- 172.21.15.129:0/152331106 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd540054f0 con 0x55c0337829a0 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 -- 172.21.15.129:0/152331106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 msgr2=0x55c033782da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd54010e50 tx=0x7fcd5400f790 comp rx=0 tx=0).stop 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 -- 172.21.15.129:0/152331106 shutdown_connections 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c033788370 0x55c03378a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c033787da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 --2- 172.21.15.129:0/152331106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337829a0 0x55c033782da0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 -- 172.21.15.129:0/152331106 >> 172.21.15.129:0/152331106 conn(0x55c03377e010 msgr2=0x55c03377e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 -- 172.21.15.129:0/152331106 shutdown_connections 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.321+0000 7fcd65a46740 1 -- 172.21.15.129:0/152331106 wait complete. 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 -- start start 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 Processor -- start 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c03378cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c03378d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c03378b2e0 con 0x55c033788370 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c03366f290 con 0x55c0337832e0 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c03378b460 con 0x55c0337829a0 2024-09-05T01:17:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd65a46740 10 client.0 _ll_register_callbacks cb 0x55c0336a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd61242640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd61242640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49572/0 (socket says 172.21.15.129:49572) 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd61242640 1 -- 172.21.15.129:0/1827468322 learned_addr learned my addr 172.21.15.129:0/1827468322 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.322+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c03378d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c03378cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 -- 172.21.15.129:0/1827468322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 msgr2=0x55c03378cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c03378cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 -- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 msgr2=0x55c03378d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c03378d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 -- 172.21.15.129:0/1827468322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0333f9210 con 0x55c033788370 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c03378d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c03378cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd61242640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcd54005bd0 tx=0x7fcd54010aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1129eee1eb5d0696 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54005230 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fcd540053d0 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd54009e00 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd5403c050 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd65a46740 10 client.15591 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0334c5620 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd54009e00 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fcd54026a60 con 0x55c033788370 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd65a46740 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.323+0000 7fcd65a46740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd65a46740 20 client.15591 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033798560 0x55c03379a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd65a46740 4 client.15591 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0336f3a20 con 0x55c033798560 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033798560 0x55c03379a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd3bfff640 10 client.15591 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.324+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033798560 0x55c03379a950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296814ae3aabb211 server_cookie=c046285c7af2144e in_seq=0 out_seq=0 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1827468322 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0336f3a20 con 0x55c033798560 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd3bfff640 10 client.15591 handle_command_reply: tid=0 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 client.15591 shutdown 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 msgr2=0x55c033794080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcd54005bd0 tx=0x7fcd54010aa0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 shutdown_connections 2024-09-05T01:17:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033798560 0x55c03379a950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c033788370 0x55c033794080 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0337832e0 0x55c03378d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd3bfff640 0 client.15591 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1827468322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c03378cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 >> 172.21.15.129:0/1827468322 conn(0x55c03377e010 msgr2=0x55c03377f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 shutdown_connections 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 1 -- 172.21.15.129:0/1827468322 wait complete. 2024-09-05T01:17:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.325+0000 7fcd65a46740 20 client.15591 trim_cache size 0 max 16384 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 Processor -- start 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 -- start start 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033782d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033787b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1ad1a0 con 0x55c0337832e0 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1aaa70 con 0x55c033788370 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd65a46740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1a88c0 con 0x55c0337829a0 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49580/0 (socket says 172.21.15.129:49580) 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 -- 172.21.15.129:0/3523638094 learned_addr learned my addr 172.21.15.129:0/3523638094 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd60a41640 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033782d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 -- 172.21.15.129:0/3523638094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 msgr2=0x55c033782d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033782d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 -- 172.21.15.129:0/3523638094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 msgr2=0x55c033787b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033787b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 -- 172.21.15.129:0/3523638094 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03378d250 con 0x55c0337832e0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd60a41640 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033782d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.340+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cd190 tx=0x7fcd48097e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a54e4dc4dae25a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd467fc640 1 -- 172.21.15.129:0/3523638094 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480a47f0 con 0x55c0337832e0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd467fc640 1 -- 172.21.15.129:0/3523638094 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd48099e80 con 0x55c0337832e0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd467fc640 1 -- 172.21.15.129:0/3523638094 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd4809b2a0 con 0x55c0337832e0 2024-09-05T01:17:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- 172.21.15.129:0/3523638094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 msgr2=0x55c033794040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cd190 tx=0x7fcd48097e60 comp rx=0 tx=0).stop 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- 172.21.15.129:0/3523638094 shutdown_connections 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033787b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033794040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 --2- 172.21.15.129:0/3523638094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033782d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- 172.21.15.129:0/3523638094 >> 172.21.15.129:0/3523638094 conn(0x55c03377e010 msgr2=0x55c03377d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- 172.21.15.129:0/3523638094 shutdown_connections 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- 172.21.15.129:0/3523638094 wait complete. 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.341+0000 7fcd65a46740 1 -- start start 2024-09-05T01:17:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 Processor -- start 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033791690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033792be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1a8f30 con 0x55c0337832e0 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1aa3a0 con 0x55c033788370 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c1a9600 con 0x55c0337829a0 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd65a46740 10 client.0 _ll_register_callbacks cb 0x7fcd48005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49592/0 (socket says 172.21.15.129:49592) 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 -- 172.21.15.129:0/1227531149 learned_addr learned my addr 172.21.15.129:0/1227531149 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:37.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033791690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 -- 172.21.15.129:0/1227531149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 msgr2=0x55c033791690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033791690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 -- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 msgr2=0x55c033792be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033792be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd5bfff640 1 -- 172.21.15.129:0/1227531149 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0337931f0 con 0x55c0337832e0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.342+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033791690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd5bfff640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcd48097fe0 tx=0x7fcd4809b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df83cc890dc7b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd480981f0 con 0x55c0337832e0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fcd480ba320 con 0x55c0337832e0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd48099e80 con 0x55c0337832e0 2024-09-05T01:17:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcd48098390 con 0x55c0337832e0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 10 client.15597 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0338031e0 con 0x55c0337832e0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd48099e80 con 0x55c0337832e0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fcd480978d0 con 0x55c0337832e0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 10 client.15597 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 20 client.15597 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033803980 0x55c033805da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 4 client.15597 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0336edd70 con 0x55c033803980 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.343+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033803980 0x55c033805da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.344+0000 7fcd60a41640 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033803980 0x55c033805da0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f81968f4b824c22 server_cookie=6d38ec71b3742596 in_seq=0 out_seq=0 2024-09-05T01:17:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.344+0000 7fcd3bfff640 10 client.15597 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.363+0000 7fcd3bfff640 1 -- 172.21.15.129:0/1227531149 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0336edd70 con 0x55c033803980 2024-09-05T01:17:37.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.363+0000 7fcd3bfff640 10 client.15597 handle_command_reply: tid=0 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.363+0000 7fcd65a46740 1 client.15597 shutdown 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 msgr2=0x55c033791bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcd48097fe0 tx=0x7fcd4809b950 comp rx=0 tx=0).stop 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 shutdown_connections 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c033803980 0x55c033805da0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c033788370 0x55c033792be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c0337832e0 0x55c033791bd0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd3bfff640 0 client.15597 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 --2- 172.21.15.129:0/1227531149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c0337829a0 0x55c033791690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 >> 172.21.15.129:0/1227531149 conn(0x55c03377e010 msgr2=0x55c03377d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 shutdown_connections 2024-09-05T01:17:37.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 1 -- 172.21.15.129:0/1227531149 wait complete. 2024-09-05T01:17:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd65a46740 20 client.15597 trim_cache size 0 max 16384 2024-09-05T01:17:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:17:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:17:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:17:37.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 msgr2=0x7fcd4c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 0x7fcd4c0a5170 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400fbd0 tx=0x7fcd5401e000 comp rx=0 tx=0).stop 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 msgr2=0x7fcd5c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.364+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9f0 tx=0x7fcd4800eeb0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 shutdown_connections 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcd4c0a2cb0 0x7fcd4c0a5170 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcd5c102600 0x7fcd5c19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd5c101c40 0x7fcd5c19ca50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 --2- 172.21.15.129:0/2755216083 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcd5c100a50 0x7fcd5c06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 >> 172.21.15.129:0/2755216083 conn(0x7fcd5c0fc1b0 msgr2=0x7fcd5c105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 shutdown_connections 2024-09-05T01:17:37.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:37.365+0000 7fcd62ca5640 1 -- 172.21.15.129:0/2755216083 wait complete. 2024-09-05T01:17:39.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:38 smithi183 ceph-mon[34589]: pgmap v420: 129 pgs: 129 active+clean; 937 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 984 op/s 2024-09-05T01:17:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:38 smithi129 ceph-mon[30979]: pgmap v420: 129 pgs: 129 active+clean; 937 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 984 op/s 2024-09-05T01:17:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:38 smithi194 ceph-mon[34941]: pgmap v420: 129 pgs: 129 active+clean; 937 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.0 MiB/s wr, 984 op/s 2024-09-05T01:17:40.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:17:41.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:40 smithi183 ceph-mon[34589]: pgmap v421: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 879 op/s 2024-09-05T01:17:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:40 smithi129 ceph-mon[30979]: pgmap v421: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 879 op/s 2024-09-05T01:17:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:40 smithi194 ceph-mon[34941]: pgmap v421: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.8 MiB/s wr, 879 op/s 2024-09-05T01:17:43.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:42 smithi183 ceph-mon[34589]: pgmap v422: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 590 op/s 2024-09-05T01:17:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:42 smithi129 ceph-mon[30979]: pgmap v422: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 590 op/s 2024-09-05T01:17:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:42 smithi194 ceph-mon[34941]: pgmap v422: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 590 op/s 2024-09-05T01:17:45.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:44 smithi183 ceph-mon[34589]: pgmap v423: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-09-05T01:17:45.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 40.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:17:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:44 smithi129 ceph-mon[30979]: pgmap v423: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-09-05T01:17:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:44 smithi194 ceph-mon[34941]: pgmap v423: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.1 MiB/s wr, 571 op/s 2024-09-05T01:17:46.643 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:17:46.904 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:17:46.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:46 smithi129 ceph-mon[30979]: pgmap v424: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-09-05T01:17:47.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:46 smithi183 ceph-mon[34589]: pgmap v424: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-09-05T01:17:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:46 smithi194 ceph-mon[34941]: pgmap v424: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.3 MiB/s wr, 245 op/s 2024-09-05T01:17:47.394 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 -- 172.21.15.129:0/990437791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260102790 msgr2=0x7f52601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 --2- 172.21.15.129:0/990437791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260102790 0x7f52601067e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5254009900 tx=0x7f525402a410 comp rx=0 tx=0).stop 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 -- 172.21.15.129:0/990437791 shutdown_connections 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 --2- 172.21.15.129:0/990437791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260106f10 0x7f526010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 --2- 172.21.15.129:0/990437791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260102790 0x7f52601067e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 --2- 172.21.15.129:0/990437791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 0x7f52601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 -- 172.21.15.129:0/990437791 >> 172.21.15.129:0/990437791 conn(0x7f52600fd8c0 msgr2=0x7f52600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 -- 172.21.15.129:0/990437791 shutdown_connections 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.372+0000 7f52665a2640 1 -- 172.21.15.129:0/990437791 wait complete. 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 Processor -- start 2024-09-05T01:17:47.395 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 -- start start 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 0x7f5260198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260102790 0x7f52601989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5260111e40 con 0x7f5260106f10 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5260111cc0 con 0x7f5260101de0 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f52665a2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5260111fc0 con 0x7f5260102790 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f5264b3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f5264b3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54866/0 (socket says 172.21.15.129:54866) 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f5264b3f640 1 -- 172.21.15.129:0/2169625888 learned_addr learned my addr 172.21.15.129:0/2169625888 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f525f7fe640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260102790 0x7f52601989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.373+0000 7f525ffff640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 0x7f5260198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:47.396 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 -- 172.21.15.129:0/2169625888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260102790 msgr2=0x7f52601989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260102790 0x7f52601989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 -- 172.21.15.129:0/2169625888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 msgr2=0x7f5260198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 0x7f5260198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 -- 172.21.15.129:0/2169625888 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526019d410 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f5264b3f640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f525800b970 tx=0x7f525800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258019070 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5258014440 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526019d700 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580075d0 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.374+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52601a4fe0 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.375+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52601021c0 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.376+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f525801c070 con 0x7f5260106f10 2024-09-05T01:17:47.397 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.377+0000 7f525d7fa640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 0x7f52480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:47.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.377+0000 7f525ffff640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 0x7f52480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:47.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.377+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f52580cead0 con 0x7f5260106f10 2024-09-05T01:17:47.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.377+0000 7f525ffff640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 0x7f52480a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:47.398 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.381+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f525809b620 con 0x7f5260106f10 2024-09-05T01:17:47.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.579+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f526010a0f0 con 0x7f5260106f10 2024-09-05T01:17:47.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.580+0000 7f525d7fa640 1 -- 172.21.15.129:0/2169625888 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11931 (secure 0 0 0) 0x7f525809af20 con 0x7f5260106f10 2024-09-05T01:17:47.581 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:17:47.581 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":27,"btime":"2024-09-05T01:15:41:859752+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:15:41.039933+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/979076613","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":979076613},{"type":"v1","addr":"172.21.15.194:6837","nonce":979076613}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24451,24452,24488]},"id":1}]} 2024-09-05T01:17:47.581 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 27 2024-09-05T01:17:47.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 msgr2=0x7f52480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:47.582 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 0x7f52480a5040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5250005fd0 tx=0x7f5250005850 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 msgr2=0x7f526019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f525800b970 tx=0x7f525800be30 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 shutdown_connections 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52480a2b80 0x7f52480a5040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5260106f10 0x7f526019cd30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5260102790 0x7f52601989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 --2- 172.21.15.129:0/2169625888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5260101de0 0x7f5260198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 >> 172.21.15.129:0/2169625888 conn(0x7f52600fd8c0 msgr2=0x7f52600fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 shutdown_connections 2024-09-05T01:17:47.583 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:47.582+0000 7f52665a2640 1 -- 172.21.15.129:0/2169625888 wait complete. 2024-09-05T01:17:48.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:47 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2169625888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2169625888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:47 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2169625888' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:17:48.313 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 9ec3ddd2 --release --await' 2024-09-05T01:17:48.313 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 9ec3ddd2 --release --await 2024-09-05T01:17:48.562 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:17:48.946 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:48 smithi129 ceph-mon[30979]: pgmap v425: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- 172.21.15.129:0/3596175944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 msgr2=0x7f3b5c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3596175944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c10da40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009ca0 tx=0x7f3b5402a950 comp rx=0 tx=0).stop 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- 172.21.15.129:0/3596175944 shutdown_connections 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3596175944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c10da40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3596175944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c102570 0x7f3b5c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3596175944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c100e20 0x7f3b5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- 172.21.15.129:0/3596175944 >> 172.21.15.129:0/3596175944 conn(0x7f3b5c0fc7d0 msgr2=0x7f3b5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- 172.21.15.129:0/3596175944 shutdown_connections 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- 172.21.15.129:0/3596175944 wait complete. 2024-09-05T01:17:49.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 Processor -- start 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.034+0000 7f3b6156c640 1 -- start start 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c100e20 0x7f3b5c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 0x7f3b5c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c110e30 con 0x7f3b5c102ef0 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c110cb0 con 0x7f3b5c100e20 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c110fb0 con 0x7f3b5c102570 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54888/0 (socket says 172.21.15.129:54888) 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 -- 172.21.15.129:0/3225666338 learned_addr learned my addr 172.21.15.129:0/3225666338 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5affd640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c100e20 0x7f3b5c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.039 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b53fff640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 0x7f3b5c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 -- 172.21.15.129:0/3225666338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 msgr2=0x7f3b5c197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 0x7f3b5c197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 -- 172.21.15.129:0/3225666338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c100e20 msgr2=0x7f3b5c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c100e20 0x7f3b5c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 -- 172.21.15.129:0/3225666338 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5c19c200 con 0x7f3b5c102ef0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b53fff640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 0x7f3b5c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b5b7fe640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3b5402ae70 tx=0x7f3b5402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b58ff9640 1 -- 172.21.15.129:0/3225666338 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54034040 con 0x7f3b5c102ef0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b58ff9640 1 -- 172.21.15.129:0/3225666338 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b540308f0 con 0x7f3b5c102ef0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5c19c490 con 0x7f3b5c102ef0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.035+0000 7f3b58ff9640 1 -- 172.21.15.129:0/3225666338 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b54004780 con 0x7f3b5c102ef0 2024-09-05T01:17:49.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.036+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b5c1a3dd0 con 0x7f3b5c102ef0 2024-09-05T01:17:49.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.038+0000 7f3b58ff9640 1 -- 172.21.15.129:0/3225666338 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3b54037020 con 0x7f3b5c102ef0 2024-09-05T01:17:49.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.038+0000 7f3b63945740 1 Processor -- start 2024-09-05T01:17:49.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 -- start start 2024-09-05T01:17:49.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b9be570620 con 0x55b9be60d270 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b9be517e50 con 0x55b9be60db90 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b63945740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9be56c5e0 con 0x55b9be618450 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54894/0 (socket says 172.21.15.129:54894) 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 -- 172.21.15.129:0/2970618641 learned_addr learned my addr 172.21.15.129:0/2970618641 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b58ff9640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 0x7f3b480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b5affd640 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b5b7fe640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 0x7f3b480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 -- 172.21.15.129:0/2970618641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 msgr2=0x55b9be61a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 -- 172.21.15.129:0/2970618641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 msgr2=0x55b9be617f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.039+0000 7f3b53fff640 1 -- 172.21.15.129:0/2970618641 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9be508d40 con 0x55b9be60d270 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b58ff9640 1 -- 172.21.15.129:0/3225666338 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3b540f0550 con 0x7f3b5c102ef0 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b5b7fe640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 0x7f3b480a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b540edfb0 tx=0x7f3b540ae040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400ccc0 tx=0x7f3b44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2c522afcdf2460 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b517fa640 1 -- 172.21.15.129:0/2970618641 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44016070 con 0x55b9be60d270 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b517fa640 1 -- 172.21.15.129:0/2970618641 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b440047a0 con 0x55b9be60d270 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.040+0000 7f3b517fa640 1 -- 172.21.15.129:0/2970618641 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44012780 con 0x55b9be60d270 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- 172.21.15.129:0/2970618641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 msgr2=0x55b9be60d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400ccc0 tx=0x7f3b44010740 comp rx=0 tx=0).stop 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- 172.21.15.129:0/2970618641 shutdown_connections 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 --2- 172.21.15.129:0/2970618641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60d270 0x55b9be60d650 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- 172.21.15.129:0/2970618641 >> 172.21.15.129:0/2970618641 conn(0x55b9be609800 msgr2=0x55b9be609be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- 172.21.15.129:0/2970618641 shutdown_connections 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- 172.21.15.129:0/2970618641 wait complete. 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 -- start start 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.041+0000 7f3b63945740 1 Processor -- start 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be615360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be615de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b9be61b020 con 0x55b9be60db90 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b9be4f8f10 con 0x55b9be60d270 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b9be61b1a0 con 0x55b9be618450 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 10 client.0 _ll_register_callbacks cb 0x55b9be51cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54898/0 (socket says 172.21.15.129:54898) 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 -- 172.21.15.129:0/561978410 learned_addr learned my addr 172.21.15.129:0/561978410 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b53fff640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be615de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b53fff640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be615360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 -- 172.21.15.129:0/561978410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 msgr2=0x55b9be615de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be615de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 -- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 msgr2=0x55b9be615360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be615360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 -- 172.21.15.129:0/561978410 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9be27f580 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b53fff640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be615360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b5affd640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00ba20 tx=0x7f3b4c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f63392a98fde32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b427fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c01b640 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3b4c01b7e0 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.042+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b440047a0 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c01b9e0 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9be349960 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b440047a0 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3b4c019020 con 0x55b9be60db90 2024-09-05T01:17:49.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 10 client.15615 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 20 client.15615 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be628050 0x55b9be62a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 4 client.15615 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.043+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9be57db90 con 0x55b9be628050 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.044+0000 7f3b53fff640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be628050 0x55b9be62a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.044+0000 7f3b53fff640 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be628050 0x55b9be62a460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a9c43de47ed921 server_cookie=ed91ca8446c9dbfa in_seq=0 out_seq=0 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.044+0000 7f3b427fc640 10 client.15615 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b427fc640 1 -- 172.21.15.129:0/561978410 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9be57db90 con 0x55b9be628050 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b427fc640 10 client.15615 handle_command_reply: tid=0 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 client.15615 shutdown 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 msgr2=0x55b9be6158a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00ba20 tx=0x7f3b4c00bee0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 shutdown_connections 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be628050 0x55b9be62a460 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be615de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be6158a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b427fc640 0 client.15615 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 --2- 172.21.15.129:0/561978410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be615360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 >> 172.21.15.129:0/561978410 conn(0x55b9be609800 msgr2=0x55b9be606750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 shutdown_connections 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 1 -- 172.21.15.129:0/561978410 wait complete. 2024-09-05T01:17:49.053 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.045+0000 7f3b63945740 20 client.15615 trim_cache size 0 max 16384 2024-09-05T01:17:49.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 Processor -- start 2024-09-05T01:17:49.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 -- start start 2024-09-05T01:17:49.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be60d270 0x55b9be60fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1af370 con 0x55b9be618450 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1acc40 con 0x55b9be60db90 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b63945740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1aaa90 con 0x55b9be60d270 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54900/0 (socket says 172.21.15.129:54900) 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 -- 172.21.15.129:0/2192732323 learned_addr learned my addr 172.21.15.129:0/2192732323 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be60d270 0x55b9be60fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b5affd640 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 -- 172.21.15.129:0/2192732323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be60d270 msgr2=0x55b9be60fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be60d270 0x55b9be60fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 -- 172.21.15.129:0/2192732323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 msgr2=0x55b9be617230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.062+0000 7f3b53fff640 1 -- 172.21.15.129:0/2192732323 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9be60d9c0 con 0x55b9be618450 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b53fff640 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400cc90 tx=0x7f3b4400c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccc87ef7cb24cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b50ff9640 1 -- 172.21.15.129:0/2192732323 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b44016070 con 0x55b9be618450 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b50ff9640 1 -- 172.21.15.129:0/2192732323 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b440049d0 con 0x55b9be618450 2024-09-05T01:17:49.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b50ff9640 1 -- 172.21.15.129:0/2192732323 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b440052d0 con 0x55b9be618450 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 -- 172.21.15.129:0/2192732323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 msgr2=0x55b9be62fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400cc90 tx=0x7f3b4400c640 comp rx=0 tx=0).stop 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 -- 172.21.15.129:0/2192732323 shutdown_connections 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be618450 0x55b9be62fd70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60db90 0x55b9be617230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 --2- 172.21.15.129:0/2192732323 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be60d270 0x55b9be60fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.063+0000 7f3b63945740 1 -- 172.21.15.129:0/2192732323 >> 172.21.15.129:0/2192732323 conn(0x55b9be609800 msgr2=0x55b9be606750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.068 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- 172.21.15.129:0/2192732323 shutdown_connections 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- 172.21.15.129:0/2192732323 wait complete. 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- start start 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 Processor -- start 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be620bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1ab100 con 0x55b9be60db90 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1ac570 con 0x55b9be60d270 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5c1ab7d0 con 0x55b9be618450 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b63945740 10 client.0 _ll_register_callbacks cb 0x7f3b4c004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54914/0 (socket says 172.21.15.129:54914) 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b5affd640 1 -- 172.21.15.129:0/158481800 learned_addr learned my addr 172.21.15.129:0/158481800 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b53fff640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.064+0000 7f3b53fff640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be620bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 -- 172.21.15.129:0/158481800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 msgr2=0x55b9be61d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 -- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 msgr2=0x55b9be620bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be620bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 -- 172.21.15.129:0/158481800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9be61dc80 con 0x55b9be60db90 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b53fff640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b5affd640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c000c00 tx=0x7f3b4c01cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f78ee551aeb1790 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c02c070 con 0x55b9be60db90 2024-09-05T01:17:49.070 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f3b4c005030 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b440049d0 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b4c038050 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b63945740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9be693e20 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3b440049d0 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.065+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7f3b4c019040 con 0x55b9be60db90 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 10 client.15621 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 20 client.15621 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be694570 0x55b9be696990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 4 client.15621 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9ec3ddd2", "release": true, "await": true}] 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9ec3ddd2", "release": true, "await": true}) -- 0x55b9be5780a0 con 0x55b9be694570 2024-09-05T01:17:49.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b53fff640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be694570 0x55b9be696990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:49.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b53fff640 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be694570 0x55b9be696990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d26020f000af6f2 server_cookie=64b45e51686865f4 in_seq=0 out_seq=0 2024-09-05T01:17:49.072 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.066+0000 7f3b427fc640 10 client.15621 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:49.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.120+0000 7f3b427fc640 1 -- 172.21.15.129:0/158481800 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b9be5780a0 con 0x55b9be694570 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.120+0000 7f3b427fc640 10 client.15621 handle_command_reply: tid=0 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.120+0000 7f3b63945740 1 client.15621 shutdown 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 msgr2=0x55b9be61cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c000c00 tx=0x7f3b4c01cdc0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 shutdown_connections 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b9be694570 0x55b9be696990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b9be618450 0x55b9be61d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b9be60db90 0x55b9be61cef0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 --2- 172.21.15.129:0/158481800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b9be60d270 0x55b9be620bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 >> 172.21.15.129:0/158481800 conn(0x55b9be609800 msgr2=0x55b9be615300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b427fc640 0 client.15621 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 shutdown_connections 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 1 -- 172.21.15.129:0/158481800 wait complete. 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.121+0000 7f3b63945740 20 client.15621 trim_cache size 0 max 16384 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 27, 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 8, 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:17:49.123 INFO:teuthology.orchestra.run.smithi129.stdout: "9ec3ddd2": { 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 8, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 144.2, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:17:49.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:17:49.125 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:17:49.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 msgr2=0x7f3b480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 0x7f3b480a5040 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b540edfb0 tx=0x7f3b540ae040 comp rx=0 tx=0).stop 2024-09-05T01:17:49.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 msgr2=0x7f3b5c19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3b5402ae70 tx=0x7f3b5402f5d0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 shutdown_connections 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3b480a2b80 0x7f3b480a5040 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3b5c102ef0 0x7f3b5c19bb20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3b5c102570 0x7f3b5c197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 --2- 172.21.15.129:0/3225666338 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b5c100e20 0x7f3b5c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:49.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 >> 172.21.15.129:0/3225666338 conn(0x7f3b5c0fc7d0 msgr2=0x7f3b5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:49.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 shutdown_connections 2024-09-05T01:17:49.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:17:49.122+0000 7f3b6156c640 1 -- 172.21.15.129:0/3225666338 wait complete. 2024-09-05T01:17:49.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:48 smithi183 ceph-mon[34589]: pgmap v425: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-09-05T01:17:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:48 smithi194 ceph-mon[34941]: pgmap v425: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.9 MiB/s wr, 123 op/s 2024-09-05T01:17:49.981 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9ec3ddd2', total seconds elapsed: 27.5 2024-09-05T01:17:50.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 45.0 s, 284.7 tps, lat 1119.768 ms stddev 3792.274 2024-09-05T01:17:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 -- 172.21.15.129:0/4031752334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 msgr2=0x7fe2941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/4031752334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2941031d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe290009900 tx=0x7fe29002a410 comp rx=0 tx=0).stop 2024-09-05T01:17:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 -- 172.21.15.129:0/4031752334 shutdown_connections 2024-09-05T01:17:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/4031752334 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294103710 0x7fe294109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/4031752334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2941031d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/4031752334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe294101b60 0x7fe294101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.505+0000 7fe29b9bd640 1 -- 172.21.15.129:0/4031752334 >> 172.21.15.129:0/4031752334 conn(0x7fe2940fd2e0 msgr2=0x7fe2940ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.507+0000 7fe29b9bd640 1 -- 172.21.15.129:0/4031752334 shutdown_connections 2024-09-05T01:17:50.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.508+0000 7fe29b9bd640 1 -- 172.21.15.129:0/4031752334 wait complete. 2024-09-05T01:17:50.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.508+0000 7fe29b9bd640 1 Processor -- start 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 -- start start 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe294103710 0x7fe294074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410d6c0 con 0x7fe294103710 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410d540 con 0x7fe294102d50 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe29b9bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe29410d840 con 0x7fe294101b60 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe299759640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe299759640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:46472/0 (socket says 172.21.15.129:46472) 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe299759640 1 -- 172.21.15.129:0/18398318 learned_addr learned my addr 172.21.15.129:0/18398318 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:50.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.509+0000 7fe298f58640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 -- 172.21.15.129:0/18398318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 msgr2=0x7fe2940745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2940745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 -- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe294103710 msgr2=0x7fe294074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe294103710 0x7fe294074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 -- 172.21.15.129:0/18398318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29419bc20 con 0x7fe294101b60 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.510+0000 7fe299759640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e9c0 tx=0x7fe28400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.511+0000 7fe28a7fc640 1 -- 172.21.15.129:0/18398318 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28400cd80 con 0x7fe294101b60 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.511+0000 7fe28a7fc640 1 -- 172.21.15.129:0/18398318 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2840035f0 con 0x7fe294101b60 2024-09-05T01:17:50.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.511+0000 7fe28a7fc640 1 -- 172.21.15.129:0/18398318 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe284010690 con 0x7fe294101b60 2024-09-05T01:17:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.511+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29419bf10 con 0x7fe294101b60 2024-09-05T01:17:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.511+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29419c380 con 0x7fe294101b60 2024-09-05T01:17:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.516+0000 7fe29e834740 1 Processor -- start 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.516+0000 7fe29e834740 1 -- start start 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd2bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557afdc8a2f0 con 0x557afdd269a0 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557afdc32270 con 0x557afdd2c370 2024-09-05T01:17:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe29e834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557afdc862b0 con 0x557afdd272e0 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe28a7fc640 1 -- 172.21.15.129:0/18398318 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe2840037a0 con 0x7fe294101b60 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe298f58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe298f58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:38640/0 (socket says 172.21.15.129:38640) 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe298f58640 1 -- 172.21.15.129:0/2204421881 learned_addr learned my addr 172.21.15.129:0/2204421881 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe299f5a640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe299759640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd2bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.517+0000 7fe298f58640 1 -- 172.21.15.129:0/2204421881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 msgr2=0x557afdd2bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.518+0000 7fe28a7fc640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 0x7fe2800a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.518+0000 7fe28a7fc640 1 -- 172.21.15.129:0/18398318 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe2840cf7a0 con 0x7fe294101b60 2024-09-05T01:17:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe298f58640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd2bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe298f58640 1 -- 172.21.15.129:0/2204421881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 msgr2=0x557afdd26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe298f58640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe298f58640 1 -- 172.21.15.129:0/2204421881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afdc31340 con 0x557afdd2c370 2024-09-05T01:17:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe299759640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:17:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe299f5a640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:17:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.519+0000 7fe298f58640 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe290009900 tx=0x7fe290004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23f4ecca39f87892 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.520+0000 7fe299f5a640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 0x7fe2800a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.520+0000 7fe27f7fe640 1 -- 172.21.15.129:0/2204421881 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290035030 con 0x557afdd2c370 2024-09-05T01:17:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.520+0000 7fe299f5a640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 0x7fe2800a5260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005fd0 tx=0x7fe28c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.520+0000 7fe27f7fe640 1 -- 172.21.15.129:0/2204421881 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe290040070 con 0x557afdd2c370 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.520+0000 7fe27f7fe640 1 -- 172.21.15.129:0/2204421881 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290031b30 con 0x557afdd2c370 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.521+0000 7fe29e834740 1 -- 172.21.15.129:0/2204421881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 msgr2=0x557afdd2e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.521+0000 7fe29e834740 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe290009900 tx=0x7fe290004170 comp rx=0 tx=0).stop 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 -- 172.21.15.129:0/2204421881 shutdown_connections 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd2e830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd2bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 --2- 172.21.15.129:0/2204421881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 -- 172.21.15.129:0/2204421881 >> 172.21.15.129:0/2204421881 conn(0x557afdd22010 msgr2=0x557afdd22410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 -- 172.21.15.129:0/2204421881 shutdown_connections 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.522+0000 7fe29e834740 1 -- 172.21.15.129:0/2204421881 wait complete. 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 -- start start 2024-09-05T01:17:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 Processor -- start 2024-09-05T01:17:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557afdd2f2e0 con 0x557afdd269a0 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557afdc13290 con 0x557afdd2c370 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557afdd2f460 con 0x557afdd272e0 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.523+0000 7fe29e834740 10 client.0 _ll_register_callbacks cb 0x557afdc44140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe298f58640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:38656/0 (socket says 172.21.15.129:38656) 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 -- 172.21.15.129:0/3511362455 learned_addr learned my addr 172.21.15.129:0/3511362455 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe298f58640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:46498/0 (socket says 172.21.15.129:46498) 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299759640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 -- 172.21.15.129:0/3511362455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 msgr2=0x557afdd311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 -- 172.21.15.129:0/3511362455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 msgr2=0x557afdd30c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 -- 172.21.15.129:0/3511362455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afd99d210 con 0x557afdd2c370 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe299f5a640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe28c007ae0 tx=0x7fe28c007470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=502bff5d3065c0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.524+0000 7fe27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c022020 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe290040070 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe28c01e450 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe28c02d050 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe29e834740 10 client.25178 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557afda69620 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe290040070 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.525+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe28c00f040 con 0x557afdd2c370 2024-09-05T01:17:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 10 client.25178 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 10 client.25178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 20 client.25178 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdd3c870 0x557afdd3ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 4 client.25178 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557afdc97a20 con 0x557afdd3c870 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe299759640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdd3c870 0x557afdd3ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe27cff9640 10 client.25178 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.526+0000 7fe299759640 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdd3c870 0x557afdd3ec80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f88c06127f9909ee server_cookie=160c5761b048c099 in_seq=0 out_seq=0 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.527+0000 7fe27cff9640 1 -- 172.21.15.129:0/3511362455 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557afdc97a20 con 0x557afdd3c870 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.527+0000 7fe27cff9640 10 client.25178 handle_command_reply: tid=0 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.527+0000 7fe29e834740 1 client.25178 shutdown 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.528+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 msgr2=0x557afdd38190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.528+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe28c007ae0 tx=0x7fe28c007470 comp rx=0 tx=0).stop 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 shutdown_connections 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdd3c870 0x557afdd3ec80 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd2c370 0x557afdd38190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd272e0 0x557afdd311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 --2- 172.21.15.129:0/3511362455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 >> 172.21.15.129:0/3511362455 conn(0x557afdd22010 msgr2=0x557afdd23820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe27cff9640 0 client.25178 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:50.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 shutdown_connections 2024-09-05T01:17:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 1 -- 172.21.15.129:0/3511362455 wait complete. 2024-09-05T01:17:50.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.529+0000 7fe29e834740 20 client.25178 trim_cache size 0 max 16384 2024-09-05T01:17:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.558+0000 7fe29e834740 1 Processor -- start 2024-09-05T01:17:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.558+0000 7fe29e834740 1 -- start start 2024-09-05T01:17:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.558+0000 7fe29e834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.558+0000 7fe29e834740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd272e0 0x557afdd38150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe299759640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe299759640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:38664/0 (socket says 172.21.15.129:38664) 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe299759640 1 -- 172.21.15.129:0/1528264900 learned_addr learned my addr 172.21.15.129:0/1528264900 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe29e834740 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd2c370 0x557afdd311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941bbf10 con 0x557afdd272e0 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe299f5a640 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd2c370 0x557afdd311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941b97e0 con 0x557afdd269a0 2024-09-05T01:17:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.559+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941a3d50 con 0x557afdd2c370 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 -- 172.21.15.129:0/1528264900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd2c370 msgr2=0x557afdd311c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd2c370 0x557afdd311c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 -- 172.21.15.129:0/1528264900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd272e0 msgr2=0x557afdd38150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd272e0 0x557afdd38150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 -- 172.21.15.129:0/1528264900 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557afdd280b0 con 0x557afdd269a0 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.560+0000 7fe299759640 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe284098050 tx=0x7fe2840046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5912373a11b10e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.561+0000 7fe27effd640 1 -- 172.21.15.129:0/1528264900 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28409bbf0 con 0x557afdd269a0 2024-09-05T01:17:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.561+0000 7fe27effd640 1 -- 172.21.15.129:0/1528264900 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe28409ffa0 con 0x557afdd269a0 2024-09-05T01:17:50.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.561+0000 7fe27effd640 1 -- 172.21.15.129:0/1528264900 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2840cf430 con 0x557afdd269a0 2024-09-05T01:17:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.562+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 msgr2=0x557afdd30c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe284098050 tx=0x7fe2840046c0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 shutdown_connections 2024-09-05T01:17:50.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd2c370 0x557afdd311c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd272e0 0x557afdd38150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 --2- 172.21.15.129:0/1528264900 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd269a0 0x557afdd30c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.563+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 >> 172.21.15.129:0/1528264900 conn(0x557afdd22010 msgr2=0x557afdd285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.564+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 shutdown_connections 2024-09-05T01:17:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.564+0000 7fe29e834740 1 -- 172.21.15.129:0/1528264900 wait complete. 2024-09-05T01:17:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.564+0000 7fe29e834740 1 -- start start 2024-09-05T01:17:50.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.564+0000 7fe29e834740 1 Processor -- start 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.565+0000 7fe29e834740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd269a0 0x557afdd350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.565+0000 7fe29e834740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.565+0000 7fe29e834740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd2c370 0x557afdda5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe29e834740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941b7e00 con 0x557afdd2c370 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe29e834740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941b9110 con 0x557afdd272e0 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe298f58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe29e834740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941b8370 con 0x557afdd269a0 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe298f58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:38672/0 (socket says 172.21.15.129:38672) 2024-09-05T01:17:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe298f58640 1 -- 172.21.15.129:0/1289522682 learned_addr learned my addr 172.21.15.129:0/1289522682 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe29e834740 10 client.0 _ll_register_callbacks cb 0x7fe28c00ae20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe299759640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd269a0 0x557afdd350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe299f5a640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd2c370 0x557afdda5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe298f58640 1 -- 172.21.15.129:0/1289522682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd269a0 msgr2=0x557afdd350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.566+0000 7fe298f58640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd269a0 0x557afdd350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe298f58640 1 -- 172.21.15.129:0/1289522682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd2c370 msgr2=0x557afdda5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe298f58640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd2c370 0x557afdda5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe298f58640 1 -- 172.21.15.129:0/1289522682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557afdda5ac0 con 0x557afdd272e0 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe298f58640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x557afdd22e90 tx=0x7fe29002ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48413c21c4a1d6e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:17:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe27cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:17:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe290053060 con 0x557afdd272e0 2024-09-05T01:17:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7fe290031a80 con 0x557afdd272e0 2024-09-05T01:17:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.567+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe28409ffa0 con 0x557afdd272e0 2024-09-05T01:17:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.569+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe29005a050 con 0x557afdd272e0 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.569+0000 7fe29e834740 10 client.25184 fetch_fsmap learned FSMap version 27 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.569+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557afdda6e40 con 0x557afdd272e0 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.569+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe28409ffa0 con 0x557afdd272e0 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.569+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8358+0+0 (secure 0 0 0) 0x7fe29004e480 con 0x557afdd272e0 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 10 client.25184 fetch_fsmap finished waiting for FSMap version 27 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:17:50.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 20 client.25184 populate_metadata read hostname 'smithi129' 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdda72c0 0x557afdda96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 4 client.25184 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557afdc91d70 con 0x557afdda72c0 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe299759640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdda72c0 0x557afdda96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.570+0000 7fe27cff9640 10 client.25184 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:50.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.571+0000 7fe299759640 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdda72c0 0x557afdda96b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d06251958c2e7bf4 server_cookie=333ea571dec5e0d1 in_seq=0 out_seq=0 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe27cff9640 1 -- 172.21.15.129:0/1289522682 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557afdc91d70 con 0x557afdda72c0 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe27cff9640 10 client.25184 handle_command_reply: tid=0 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 client.25184 shutdown 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 msgr2=0x557afdd355f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x557afdd22e90 tx=0x7fe29002ab90 comp rx=0 tx=0).stop 2024-09-05T01:17:50.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 shutdown_connections 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557afdda72c0 0x557afdda96b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557afdd2c370 0x557afdda5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557afdd272e0 0x557afdd355f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 --2- 172.21.15.129:0/1289522682 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557afdd269a0 0x557afdd350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 >> 172.21.15.129:0/1289522682 conn(0x557afdd22010 msgr2=0x557afdd285c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.591+0000 7fe27cff9640 0 client.25184 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.592+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 shutdown_connections 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.592+0000 7fe29e834740 1 -- 172.21.15.129:0/1289522682 wait complete. 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.593+0000 7fe29e834740 20 client.25184 trim_cache size 0 max 16384 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:17:50.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 msgr2=0x7fe2800a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 0x7fe2800a5260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe28c005fd0 tx=0x7fe28c004a40 comp rx=0 tx=0).stop 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 msgr2=0x7fe294075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe28400e9c0 tx=0x7fe28400ee80 comp rx=0 tx=0).stop 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 shutdown_connections 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe2800a2da0 0x7fe2800a5260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe294103710 0x7fe294074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.596 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe294102d50 0x7fe2940745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 --2- 172.21.15.129:0/18398318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe294101b60 0x7fe294075fa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:17:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.595+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 >> 172.21.15.129:0/18398318 conn(0x7fe2940fd2e0 msgr2=0x7fe2941060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:17:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.596+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 shutdown_connections 2024-09-05T01:17:50.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:17:50.596+0000 7fe29b9bd640 1 -- 172.21.15.129:0/18398318 wait complete. 2024-09-05T01:17:51.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:50 smithi183 ceph-mon[34589]: pgmap v426: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-09-05T01:17:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:50 smithi129 ceph-mon[30979]: pgmap v426: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-09-05T01:17:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:50 smithi194 ceph-mon[34941]: pgmap v426: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 MiB/s wr, 64 op/s 2024-09-05T01:17:53.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:52 smithi183 ceph-mon[34589]: pgmap v427: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 391 KiB/s wr, 51 op/s 2024-09-05T01:17:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:52 smithi129 ceph-mon[30979]: pgmap v427: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 391 KiB/s wr, 51 op/s 2024-09-05T01:17:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:52 smithi194 ceph-mon[34941]: pgmap v427: 129 pgs: 129 active+clean; 945 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 391 KiB/s wr, 51 op/s 2024-09-05T01:17:55.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:54 smithi183 ceph-mon[34589]: pgmap v428: 129 pgs: 129 active+clean; 945 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-05T01:17:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 50.0 s, 1168.0 tps, lat 85.762 ms stddev 24.093 2024-09-05T01:17:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:54 smithi129 ceph-mon[30979]: pgmap v428: 129 pgs: 129 active+clean; 945 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-05T01:17:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:54 smithi194 ceph-mon[34941]: pgmap v428: 129 pgs: 129 active+clean; 945 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 2.1 MiB/s wr, 270 op/s 2024-09-05T01:17:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:56 smithi129 ceph-mon[30979]: pgmap v429: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-05T01:17:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:56 smithi194 ceph-mon[34941]: pgmap v429: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-05T01:17:57.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:56 smithi183 ceph-mon[34589]: pgmap v429: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.1 MiB/s wr, 477 op/s 2024-09-05T01:17:58.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:57 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:17:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:57 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:17:58.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:57 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:17:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:17:58 smithi129 ceph-mon[30979]: pgmap v430: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 646 op/s 2024-09-05T01:17:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:17:58 smithi194 ceph-mon[34941]: pgmap v430: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 646 op/s 2024-09-05T01:17:59.311 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:17:58 smithi183 ceph-mon[34589]: pgmap v430: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 646 op/s 2024-09-05T01:18:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 55.0 s, 1198.6 tps, lat 83.435 ms stddev 45.330 2024-09-05T01:18:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:00 smithi129 ceph-mon[30979]: pgmap v431: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-09-05T01:18:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:00 smithi194 ceph-mon[34941]: pgmap v431: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-09-05T01:18:01.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:00 smithi183 ceph-mon[34589]: pgmap v431: 129 pgs: 129 active+clean; 945 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 8.5 MiB/s wr, 1.01k op/s 2024-09-05T01:18:03.237 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:02 smithi129 ceph-mon[30979]: pgmap v432: 129 pgs: 129 active+clean; 945 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-09-05T01:18:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:02 smithi194 ceph-mon[34941]: pgmap v432: 129 pgs: 129 active+clean; 945 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-09-05T01:18:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:02 smithi183 ceph-mon[34589]: pgmap v432: 129 pgs: 129 active+clean; 945 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 1.05k op/s 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 -- 172.21.15.129:0/2919147194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac105900 msgr2=0x7f65ac10c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 --2- 172.21.15.129:0/2919147194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac105900 0x7f65ac10c190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009be0 tx=0x7f65a402a710 comp rx=0 tx=0).stop 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 -- 172.21.15.129:0/2919147194 shutdown_connections 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 --2- 172.21.15.129:0/2919147194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac105900 0x7f65ac10c190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 --2- 172.21.15.129:0/2919147194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac104f40 0x7f65ac1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 --2- 172.21.15.129:0/2919147194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac103d50 0x7f65ac104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.740+0000 7f65b2903640 1 -- 172.21.15.129:0/2919147194 >> 172.21.15.129:0/2919147194 conn(0x7f65ac0ff4b0 msgr2=0x7f65ac1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.741+0000 7f65b2903640 1 -- 172.21.15.129:0/2919147194 shutdown_connections 2024-09-05T01:18:03.743 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.741+0000 7f65b2903640 1 -- 172.21.15.129:0/2919147194 wait complete. 2024-09-05T01:18:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.741+0000 7f65b2903640 1 Processor -- start 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.741+0000 7f65b2903640 1 -- start start 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac103d50 0x7f65ac06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 0x7f65ac06c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac10f860 con 0x7f65ac103d50 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac10f6e0 con 0x7f65ac105900 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b2903640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac10f9e0 con 0x7f65ac104f40 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b0ea0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b0ea0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46002/0 (socket says 172.21.15.129:46002) 2024-09-05T01:18:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65b0ea0640 1 -- 172.21.15.129:0/3844599611 learned_addr learned my addr 172.21.15.129:0/3844599611 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.742+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 0x7f65ac06c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 -- 172.21.15.129:0/3844599611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 msgr2=0x7f65ac06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 0x7f65ac06c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 -- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac103d50 msgr2=0x7f65ac06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac103d50 0x7f65ac06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 -- 172.21.15.129:0/3844599611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac073c10 con 0x7f65ac105900 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 0x7f65ac06c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65a4030970 tx=0x7f65a4031440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65a97fa640 1 -- 172.21.15.129:0/3844599611 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4036040 con 0x7f65ac105900 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65a97fa640 1 -- 172.21.15.129:0/3844599611 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a4031830 con 0x7f65ac105900 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65a97fa640 1 -- 172.21.15.129:0/3844599611 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a404a3e0 con 0x7f65ac105900 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.743+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac073ea0 con 0x7f65ac105900 2024-09-05T01:18:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.744+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac076630 con 0x7f65ac105900 2024-09-05T01:18:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.748+0000 7f65a97fa640 1 -- 172.21.15.129:0/3844599611 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f65a404a580 con 0x7f65ac105900 2024-09-05T01:18:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.749+0000 7f65a97fa640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 0x7f65940a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.749+0000 7f65a97fa640 1 -- 172.21.15.129:0/3844599611 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f65a40f9610 con 0x7f65ac105900 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65abfff640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 0x7f65940a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 Processor -- start 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 -- start start 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b4bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b4e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b094aaa2f0 con 0x55b094b4c370 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b094a52270 con 0x55b094b472e0 2024-09-05T01:18:03.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.750+0000 7f65b5699740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b094aa62b0 con 0x55b094b469a0 2024-09-05T01:18:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.751+0000 7f65abfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.751+0000 7f65abfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:34288/0 (socket says 172.21.15.129:34288) 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.751+0000 7f65abfff640 1 -- 172.21.15.129:0/1825279775 learned_addr learned my addr 172.21.15.129:0/1825279775 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.751+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b4e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.751+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b4bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 -- 172.21.15.129:0/1825279775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 msgr2=0x55b094b46da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 -- 172.21.15.129:0/1825279775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 msgr2=0x55b094b4bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b4bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 -- 172.21.15.129:0/1825279775 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b094a51340 con 0x55b094b4c370 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65abfff640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 0x7f65940a52e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f65ac104da0 tx=0x7f659c0127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b4e830 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f65a40c5990 tx=0x7f65a4052300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f55a63c82e047a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.752+0000 7f65927fc640 1 -- 172.21.15.129:0/1825279775 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40de020 con 0x55b094b4c370 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.753+0000 7f65927fc640 1 -- 172.21.15.129:0/1825279775 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a40c9db0 con 0x55b094b4c370 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.753+0000 7f65927fc640 1 -- 172.21.15.129:0/1825279775 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40f9270 con 0x55b094b4c370 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- 172.21.15.129:0/1825279775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 msgr2=0x55b094b4e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b4e830 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f65a40c5990 tx=0x7f65a4052300 comp rx=0 tx=0).stop 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- 172.21.15.129:0/1825279775 shutdown_connections 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b4e830 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b4bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 --2- 172.21.15.129:0/1825279775 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- 172.21.15.129:0/1825279775 >> 172.21.15.129:0/1825279775 conn(0x55b094b42010 msgr2=0x55b094b42410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- 172.21.15.129:0/1825279775 shutdown_connections 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- 172.21.15.129:0/1825279775 wait complete. 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.754+0000 7f65b5699740 1 -- start start 2024-09-05T01:18:03.756 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 Processor -- start 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b50c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b57c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b50c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b094aaa2f0 con 0x55b094b472e0 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b094a33290 con 0x55b094b469a0 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b094b4f100 con 0x55b094b4c370 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46026/0 (socket says 172.21.15.129:46026) 2024-09-05T01:18:03.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65abfff640 1 -- 172.21.15.129:0/2091095273 learned_addr learned my addr 172.21.15.129:0/2091095273 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.755+0000 7f65b5699740 10 client.0 _ll_register_callbacks cb 0x55b094a64140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b57c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 -- 172.21.15.129:0/2091095273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 msgr2=0x55b094b57c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b57c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 -- 172.21.15.129:0/2091095273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 msgr2=0x55b094b50c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b50c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 -- 172.21.15.129:0/2091095273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0947bd210 con 0x55b094b469a0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f65abfff640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f659c01d520 tx=0x7f659c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d516146164d333bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.756+0000 7f658b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c010850 con 0x55b094b469a0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f659c0109f0 con 0x55b094b469a0 2024-09-05T01:18:03.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a40c9db0 con 0x55b094b469a0 2024-09-05T01:18:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f659c032050 con 0x55b094b469a0 2024-09-05T01:18:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f65b5699740 10 client.25196 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.757+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b094889620 con 0x55b094b469a0 2024-09-05T01:18:03.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f659c010bf0 con 0x55b094b469a0 2024-09-05T01:18:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f659c00f3c0 con 0x55b094b469a0 2024-09-05T01:18:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 10 client.25196 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 10 client.25196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:03.761 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 20 client.25196 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094b5c420 0x55b094b5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 4 client.25196 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.758+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f65a40c9db0 con 0x55b094b5c420 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.759+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094b5c420 0x55b094b5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.760+0000 7f658b7fe640 10 client.25196 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.760+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094b5c420 0x55b094b5e860 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118d9b1a894db8d1 server_cookie=ac498c66701dc70b in_seq=0 out_seq=0 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f658b7fe640 1 -- 172.21.15.129:0/2091095273 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f65a40c9db0 con 0x55b094b5c420 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f658b7fe640 10 client.25196 handle_command_reply: tid=0 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 client.25196 shutdown 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 msgr2=0x55b094b506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f659c01d520 tx=0x7f659c009260 comp rx=0 tx=0).stop 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 shutdown_connections 2024-09-05T01:18:03.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094b5c420 0x55b094b5e860 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b57c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b50c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 --2- 172.21.15.129:0/2091095273 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b506e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 >> 172.21.15.129:0/2091095273 conn(0x55b094b42010 msgr2=0x55b094b43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f658b7fe640 0 client.25196 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 shutdown_connections 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.761+0000 7f65b5699740 1 -- 172.21.15.129:0/2091095273 wait complete. 2024-09-05T01:18:03.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.762+0000 7f65b5699740 20 client.25196 trim_cache size 0 max 16384 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 Processor -- start 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 -- start start 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46036/0 (socket says 172.21.15.129:46036) 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65ab7fe640 1 -- 172.21.15.129:0/1969675127 learned_addr learned my addr 172.21.15.129:0/1969675127 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6598012690 con 0x55b094b4c370 2024-09-05T01:18:03.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f659800ff60 con 0x55b094b472e0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f659800ddb0 con 0x55b094b469a0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.789+0000 7f65abfff640 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 -- 172.21.15.129:0/1969675127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 msgr2=0x55b094b46d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 -- 172.21.15.129:0/1969675127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 msgr2=0x55b094b410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 -- 172.21.15.129:0/1969675127 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b094b50220 con 0x55b094b472e0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005e00 tx=0x7f65a00044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20564b658188255 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f6591ffb640 1 -- 172.21.15.129:0/1969675127 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0003dc0 con 0x55b094b472e0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f6591ffb640 1 -- 172.21.15.129:0/1969675127 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a001e8d0 con 0x55b094b472e0 2024-09-05T01:18:03.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.790+0000 7f6591ffb640 1 -- 172.21.15.129:0/1969675127 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0012b70 con 0x55b094b472e0 2024-09-05T01:18:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.791+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 msgr2=0x55b094b57bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.791+0000 7f65b5699740 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005e00 tx=0x7f65a00044d0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.792+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 shutdown_connections 2024-09-05T01:18:03.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.792+0000 7f65b5699740 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b4c370 0x55b094b410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.792+0000 7f65b5699740 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b472e0 0x55b094b57bf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.792+0000 7f65b5699740 1 --2- 172.21.15.129:0/1969675127 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b469a0 0x55b094b46d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.792+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 >> 172.21.15.129:0/1969675127 conn(0x55b094b42010 msgr2=0x55b094b409a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.793+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 shutdown_connections 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.793+0000 7f65b5699740 1 -- 172.21.15.129:0/1969675127 wait complete. 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.793+0000 7f65b5699740 1 -- start start 2024-09-05T01:18:03.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.794+0000 7f65b5699740 1 Processor -- start 2024-09-05T01:18:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.795+0000 7f65b5699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b556d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b55c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f659800e420 con 0x55b094b472e0 2024-09-05T01:18:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f659800f890 con 0x55b094b469a0 2024-09-05T01:18:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f659800eaf0 con 0x55b094b4c370 2024-09-05T01:18:03.797 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.796+0000 7f65b5699740 10 client.0 _ll_register_callbacks cb 0x7f659c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:34318/0 (socket says 172.21.15.129:34318) 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 -- 172.21.15.129:0/955989598 learned_addr learned my addr 172.21.15.129:0/955989598 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b55c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65abfff640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 -- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 msgr2=0x55b094b556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 -- 172.21.15.129:0/955989598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 msgr2=0x55b094b55c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:03.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b55c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65b0ea0640 1 -- 172.21.15.129:0/955989598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b094b5eee0 con 0x55b094b4c370 2024-09-05T01:18:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65abfff640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.797+0000 7f65ab7fe640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b55c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f65b0ea0640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65a4031040 tx=0x7f65a4005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0f12a4dee68123 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:03.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f658b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40de040 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f65a40e42a0 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a001e8d0 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f65a40ce670 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f65b5699740 10 client.25162 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.798+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b094bc7200 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65a001e8d0 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f65a40c5cd0 con 0x55b094b4c370 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 10 client.25162 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:03.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 10 client.25162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 20 client.25162 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094bc7b60 0x55b094bc9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 4 client.25162 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b094ab7a20 con 0x55b094bc7b60 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.799+0000 7f65abfff640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094bc7b60 0x55b094bc9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.800+0000 7f65abfff640 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094bc7b60 0x55b094bc9f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb3fd799cc0251e server_cookie=d6e5ed4f46e9b467 in_seq=0 out_seq=0 2024-09-05T01:18:03.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.800+0000 7f658b7fe640 10 client.25162 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.833+0000 7f658b7fe640 1 -- 172.21.15.129:0/955989598 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b094ab7a20 con 0x55b094bc7b60 2024-09-05T01:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.833+0000 7f658b7fe640 10 client.25162 handle_command_reply: tid=0 2024-09-05T01:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.833+0000 7f65b5699740 1 client.25162 shutdown 2024-09-05T01:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 msgr2=0x55b094b56bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f65a4031040 tx=0x7f65a4005eb0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 shutdown_connections 2024-09-05T01:18:03.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b094bc7b60 0x55b094bc9f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b094b4c370 0x55b094b56bb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b094b472e0 0x55b094b55c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f658b7fe640 0 client.25162 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 --2- 172.21.15.129:0/955989598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b094b469a0 0x55b094b556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 >> 172.21.15.129:0/955989598 conn(0x55b094b42010 msgr2=0x55b094b411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.834+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 shutdown_connections 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.835+0000 7f65b5699740 1 -- 172.21.15.129:0/955989598 wait complete. 2024-09-05T01:18:03.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.835+0000 7f65b5699740 20 client.25162 trim_cache size 0 max 16384 2024-09-05T01:18:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:18:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:18:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:18:03.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.836+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 msgr2=0x7f65940a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.836+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 0x7f65940a52e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f65ac104da0 tx=0x7f659c0127c0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.836+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 msgr2=0x7f65ac0735a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.836+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65a4030970 tx=0x7f65a4031440 comp rx=0 tx=0).stop 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.836+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 shutdown_connections 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f65940a2e20 0x7f65940a52e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f65ac105900 0x7f65ac0735a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f65ac104f40 0x7f65ac06c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 --2- 172.21.15.129:0/3844599611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65ac103d50 0x7f65ac06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 >> 172.21.15.129:0/3844599611 conn(0x7f65ac0ff4b0 msgr2=0x7f65ac108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 shutdown_connections 2024-09-05T01:18:03.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:03.837+0000 7f65b2903640 1 -- 172.21.15.129:0/3844599611 wait complete. 2024-09-05T01:18:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 60.0 s, 1225.2 tps, lat 38.461 ms stddev 25.015 2024-09-05T01:18:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:04 smithi129 ceph-mon[30979]: pgmap v433: 129 pgs: 129 active+clean; 947 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-09-05T01:18:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:04 smithi194 ceph-mon[34941]: pgmap v433: 129 pgs: 129 active+clean; 947 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-09-05T01:18:05.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:04 smithi183 ceph-mon[34589]: pgmap v433: 129 pgs: 129 active+clean; 947 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 13 MiB/s wr, 1.52k op/s 2024-09-05T01:18:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:06 smithi129 ceph-mon[30979]: pgmap v434: 129 pgs: 129 active+clean; 949 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-05T01:18:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:06 smithi194 ceph-mon[34941]: pgmap v434: 129 pgs: 129 active+clean; 949 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-05T01:18:07.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:06 smithi183 ceph-mon[34589]: pgmap v434: 129 pgs: 129 active+clean; 949 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 1.46k op/s 2024-09-05T01:18:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:08 smithi129 ceph-mon[30979]: pgmap v435: 129 pgs: 129 active+clean; 951 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.55k op/s 2024-09-05T01:18:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:08 smithi194 ceph-mon[34941]: pgmap v435: 129 pgs: 129 active+clean; 951 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.55k op/s 2024-09-05T01:18:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:08 smithi183 ceph-mon[34589]: pgmap v435: 129 pgs: 129 active+clean; 951 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 1.55k op/s 2024-09-05T01:18:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 65.0 s, 1359.4 tps, lat 112.405 ms stddev 441.762 2024-09-05T01:18:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:10 smithi129 ceph-mon[30979]: pgmap v436: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-05T01:18:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:18:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:18:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:10 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:18:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:10 smithi194 ceph-mon[34941]: pgmap v436: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-05T01:18:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:18:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:18:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:10 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:18:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:10 smithi183 ceph-mon[34589]: pgmap v436: 129 pgs: 129 active+clean; 952 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 15 MiB/s wr, 1.69k op/s 2024-09-05T01:18:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:18:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:18:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:10 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:18:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:18:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:18:12.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:18:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:12 smithi129 ceph-mon[30979]: pgmap v437: 129 pgs: 129 active+clean; 953 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-05T01:18:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:12 smithi194 ceph-mon[34941]: pgmap v437: 129 pgs: 129 active+clean; 953 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-05T01:18:13.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:12 smithi183 ceph-mon[34589]: pgmap v437: 129 pgs: 129 active+clean; 953 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 12 MiB/s wr, 1.36k op/s 2024-09-05T01:18:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 70.0 s, 1131.8 tps, lat 88.174 ms stddev 25.194 2024-09-05T01:18:15.241 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:14 smithi129 ceph-mon[30979]: pgmap v438: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-09-05T01:18:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:14 smithi194 ceph-mon[34941]: pgmap v438: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-09-05T01:18:15.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:14 smithi183 ceph-mon[34589]: pgmap v438: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 MiB/s wr, 1.75k op/s 2024-09-05T01:18:16.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:16 smithi183 ceph-mon[34589]: pgmap v439: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-05T01:18:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:16 smithi129 ceph-mon[30979]: pgmap v439: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-05T01:18:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:16 smithi194 ceph-mon[34941]: pgmap v439: 129 pgs: 129 active+clean; 953 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 12 MiB/s wr, 1.39k op/s 2024-09-05T01:18:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.975+0000 7fad07326640 1 -- 172.21.15.129:0/184070319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00104f70 msgr2=0x7fad001053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.975+0000 7fad07326640 1 --2- 172.21.15.129:0/184070319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00104f70 0x7fad001053f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7facf0009860 tx=0x7facf002a340 comp rx=0 tx=0).stop 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 -- 172.21.15.129:0/184070319 shutdown_connections 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 --2- 172.21.15.129:0/184070319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00105930 0x7fad0010c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 --2- 172.21.15.129:0/184070319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00104f70 0x7fad001053f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 --2- 172.21.15.129:0/184070319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad00103d80 0x7fad00104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 -- 172.21.15.129:0/184070319 >> 172.21.15.129:0/184070319 conn(0x7fad000ff4e0 msgr2=0x7fad00101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 -- 172.21.15.129:0/184070319 shutdown_connections 2024-09-05T01:18:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.976+0000 7fad07326640 1 -- 172.21.15.129:0/184070319 wait complete. 2024-09-05T01:18:16.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.977+0000 7fad07326640 1 Processor -- start 2024-09-05T01:18:16.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 -- start start 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad00103d80 0x7fad00195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 0x7fad00196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 0x7fad0019d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad0010f890 con 0x7fad00103d80 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad0010f710 con 0x7fad00105930 2024-09-05T01:18:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.978+0000 7fad07326640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fad0010fa10 con 0x7fad00104f70 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad058c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 0x7fad0019d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad058c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 0x7fad0019d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43094/0 (socket says 172.21.15.129:43094) 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad058c3640 1 -- 172.21.15.129:0/297392824 learned_addr learned my addr 172.21.15.129:0/297392824 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 0x7fad00196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 -- 172.21.15.129:0/297392824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 msgr2=0x7fad0019d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 0x7fad0019d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 -- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad00103d80 msgr2=0x7fad00195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad00103d80 0x7fad00195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 -- 172.21.15.129:0/297392824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad001ae790 con 0x7fad00104f70 2024-09-05T01:18:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.979+0000 7fad048c1640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 0x7fad00196060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf002a860 tx=0x7facf002ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.980+0000 7facf67fc640 1 -- 172.21.15.129:0/297392824 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0042070 con 0x7fad00104f70 2024-09-05T01:18:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.980+0000 7facf67fc640 1 -- 172.21.15.129:0/297392824 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf00055f0 con 0x7fad00104f70 2024-09-05T01:18:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.980+0000 7facf67fc640 1 -- 172.21.15.129:0/297392824 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf0037040 con 0x7fad00104f70 2024-09-05T01:18:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.980+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad001aea20 con 0x7fad00104f70 2024-09-05T01:18:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.981+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad001aeee0 con 0x7fad00104f70 2024-09-05T01:18:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.984+0000 7facf67fc640 1 -- 172.21.15.129:0/297392824 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7facf0045070 con 0x7fad00104f70 2024-09-05T01:18:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.984+0000 7fad0a0b9740 1 Processor -- start 2024-09-05T01:18:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 -- start start 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb9022f0 con 0x55a0cb99f2e0 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb8aa270 con 0x55a0cb99e9a0 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb8fe2b0 con 0x55a0cb9a4370 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43100/0 (socket says 172.21.15.129:43100) 2024-09-05T01:18:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 -- 172.21.15.129:0/26321824 learned_addr learned my addr 172.21.15.129:0/26321824 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad050c2640 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 -- 172.21.15.129:0/26321824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 msgr2=0x55a0cb9a6830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9a6830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 -- 172.21.15.129:0/26321824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 msgr2=0x55a0cb9a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.985+0000 7fad058c3640 1 -- 172.21.15.129:0/26321824 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cb8a9340 con 0x55a0cb99e9a0 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.986+0000 7fad058c3640 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facf8011570 tx=0x7facf8011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7947a4ce380d13d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.986+0000 7faceb7fe640 1 -- 172.21.15.129:0/26321824 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8004b20 con 0x55a0cb99e9a0 2024-09-05T01:18:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.986+0000 7faceb7fe640 1 -- 172.21.15.129:0/26321824 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf8004cc0 con 0x55a0cb99e9a0 2024-09-05T01:18:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.986+0000 7faceb7fe640 1 -- 172.21.15.129:0/26321824 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8005480 con 0x55a0cb99e9a0 2024-09-05T01:18:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7facf67fc640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 0x7facec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 -- 172.21.15.129:0/26321824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 msgr2=0x55a0cb99eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7facf8011570 tx=0x7facf8011a30 comp rx=0 tx=0).stop 2024-09-05T01:18:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 -- 172.21.15.129:0/26321824 shutdown_connections 2024-09-05T01:18:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7facf67fc640 1 -- 172.21.15.129:0/297392824 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7facf00eef40 con 0x7fad00104f70 2024-09-05T01:18:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad050c2640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 0x7facec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:16.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/26321824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb99eda0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.987+0000 7fad0a0b9740 1 -- 172.21.15.129:0/26321824 >> 172.21.15.129:0/26321824 conn(0x55a0cb99a010 msgr2=0x55a0cb99a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.988+0000 7fad0a0b9740 1 -- 172.21.15.129:0/26321824 shutdown_connections 2024-09-05T01:18:16.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.988+0000 7fad0a0b9740 1 -- 172.21.15.129:0/26321824 wait complete. 2024-09-05T01:18:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.988+0000 7fad050c2640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 0x7facec0a52e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55a0cb99ae90 tx=0x7facfc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.988+0000 7fad0a0b9740 1 -- start start 2024-09-05T01:18:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.988+0000 7fad0a0b9740 1 Processor -- start 2024-09-05T01:18:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9b0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad050c2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad050c2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60018/0 (socket says 172.21.15.129:60018) 2024-09-05T01:18:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad050c2640 1 -- 172.21.15.129:0/3325755609 learned_addr learned my addr 172.21.15.129:0/3325755609 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb9a7340 con 0x55a0cb9a4370 2024-09-05T01:18:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad058c3640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad048c1640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb88b290 con 0x55a0cb99f2e0 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.989+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a0cb9a74c0 con 0x55a0cb99e9a0 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad0a0b9740 10 client.0 _ll_register_callbacks cb 0x55a0cb8bc140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad048c1640 1 -- 172.21.15.129:0/3325755609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 msgr2=0x55a0cb9a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad048c1640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad048c1640 1 -- 172.21.15.129:0/3325755609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 msgr2=0x55a0cb9b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad048c1640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad048c1640 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cb615210 con 0x55a0cb99f2e0 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.990+0000 7fad058c3640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf8004cc0 con 0x55a0cb99f2e0 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7fad048c1640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a90d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf00bb030 tx=0x7facf00b75d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a702fb0ecefd0b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7face8ff9640 10 client.25211 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7face8ff9640 1 -- 172.21.15.129:0/3325755609 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf00bac90 con 0x55a0cb99f2e0 2024-09-05T01:18:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7face8ff9640 1 -- 172.21.15.129:0/3325755609 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7facf00b9630 con 0x55a0cb99f2e0 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.991+0000 7face8ff9640 1 -- 172.21.15.129:0/3325755609 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facf0004e40 con 0x55a0cb99f2e0 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 10 client.25211 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55a0cb6e1620 con 0x55a0cb99f2e0 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7face8ff9640 1 -- 172.21.15.129:0/3325755609 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facf00c3b10 con 0x55a0cb99f2e0 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 10 client.25211 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 10 client.25211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 20 client.25211 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cb9b4640 0x55a0cb9b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 4 client.25211 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cb90fa20 con 0x55a0cb9b4640 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.992+0000 7fad050c2640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cb9b4640 0x55a0cb9b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.993+0000 7face8ff9640 10 client.25211 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.993+0000 7fad050c2640 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cb9b4640 0x55a0cb9b6a50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b739ea4699a29503 server_cookie=de234384c1c207cb in_seq=0 out_seq=0 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.993+0000 7face8ff9640 1 -- 172.21.15.129:0/3325755609 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0cb90fa20 con 0x55a0cb9b4640 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.993+0000 7face8ff9640 10 client.25211 handle_command_reply: tid=0 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 client.25211 shutdown 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 msgr2=0x55a0cb9a90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a90d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf00bb030 tx=0x7facf00b75d0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 shutdown_connections 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cb9b4640 0x55a0cb9b6a50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9a90d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7face8ff9640 0 client.25211 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3325755609 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 >> 172.21.15.129:0/3325755609 conn(0x55a0cb99a010 msgr2=0x55a0cb99b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:17.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 shutdown_connections 2024-09-05T01:18:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3325755609 wait complete. 2024-09-05T01:18:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:16.994+0000 7fad0a0b9740 20 client.25211 trim_cache size 0 max 16384 2024-09-05T01:18:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.016+0000 7fad0a0b9740 1 Processor -- start 2024-09-05T01:18:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.016+0000 7fad0a0b9740 1 -- start start 2024-09-05T01:18:17.017 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad001bbda0 con 0x55a0cb99f2e0 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b9670 con 0x55a0cb99e9a0 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b74c0 con 0x55a0cb9a4370 2024-09-05T01:18:17.018 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad050c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60032/0 (socket says 172.21.15.129:60032) 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad050c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43122/0 (socket says 172.21.15.129:43122) 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 -- 172.21.15.129:0/4180618118 learned_addr learned my addr 172.21.15.129:0/4180618118 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad048c1640 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 -- 172.21.15.129:0/4180618118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 msgr2=0x55a0cb9b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9b6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 -- 172.21.15.129:0/4180618118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 msgr2=0x55a0cb9b00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9b00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.017+0000 7fad058c3640 1 -- 172.21.15.129:0/4180618118 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cb9b70a0 con 0x55a0cb9a4370 2024-09-05T01:18:17.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.018+0000 7fad058c3640 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facf8011f30 tx=0x7facf8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fc80f378f9f7709 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.018+0000 7faceaffd640 1 -- 172.21.15.129:0/4180618118 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf800f8d0 con 0x55a0cb9a4370 2024-09-05T01:18:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.018+0000 7faceaffd640 1 -- 172.21.15.129:0/4180618118 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf8005090 con 0x55a0cb9a4370 2024-09-05T01:18:17.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.018+0000 7faceaffd640 1 -- 172.21.15.129:0/4180618118 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8019690 con 0x55a0cb9a4370 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 -- 172.21.15.129:0/4180618118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 msgr2=0x55a0cb9990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7facf8011f30 tx=0x7facf8004ab0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 -- 172.21.15.129:0/4180618118 shutdown_connections 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb9a4370 0x55a0cb9990b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9b00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/4180618118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9b6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 -- 172.21.15.129:0/4180618118 >> 172.21.15.129:0/4180618118 conn(0x55a0cb99a010 msgr2=0x55a0cb9989a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 -- 172.21.15.129:0/4180618118 shutdown_connections 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.019+0000 7fad0a0b9740 1 -- 172.21.15.129:0/4180618118 wait complete. 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.020+0000 7fad0a0b9740 1 -- start start 2024-09-05T01:18:17.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.020+0000 7fad0a0b9740 1 Processor -- start 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.020+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9ae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b7b30 con 0x55a0cb9a4370 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b8fa0 con 0x55a0cb99f2e0 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fad001b8200 con 0x55a0cb99e9a0 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad0a0b9740 10 client.0 _ll_register_callbacks cb 0x7facf0009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:17.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60044/0 (socket says 172.21.15.129:60044) 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 -- 172.21.15.129:0/3777353886 learned_addr learned my addr 172.21.15.129:0/3777353886 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad048c1640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 -- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 msgr2=0x55a0cb9ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 -- 172.21.15.129:0/3777353886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 msgr2=0x55a0cb9af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 -- 172.21.15.129:0/3777353886 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cb9af830 con 0x55a0cb99e9a0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7fad050c2640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facfc005fd0 tx=0x7facfc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8aadc49102076d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.021+0000 7face8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc01f8f0 con 0x55a0cb99e9a0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7facfc01fa90 con 0x55a0cb99e9a0 2024-09-05T01:18:17.024 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facf8005090 con 0x55a0cb99e9a0 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7facfc039050 con 0x55a0cb99e9a0 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7fad0a0b9740 10 client.25177 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a0cba1f7f0 con 0x55a0cb99e9a0 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.022+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7facf8005090 con 0x55a0cb99e9a0 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7facfc00f3d0 con 0x55a0cb99e9a0 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 10 client.25177 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 10 client.25177 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:17.025 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 20 client.25177 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cba200f0 0x55a0cba22530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 4 client.25177 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0cb909d70 con 0x55a0cba200f0 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.023+0000 7fad048c1640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cba200f0 0x55a0cba22530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.024+0000 7fad048c1640 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cba200f0 0x55a0cba22530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e180d0b81a20ef0 server_cookie=6cc42d01d3bec064 in_seq=0 out_seq=0 2024-09-05T01:18:17.026 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:17.024+0000 7face8ff9640 10 client.25177 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:19.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:18 smithi183 ceph-mon[34589]: pgmap v440: 129 pgs: 129 active+clean; 965 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-05T01:18:19.193 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:18 smithi129 ceph-mon[30979]: pgmap v440: 129 pgs: 129 active+clean; 965 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-05T01:18:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:18 smithi194 ceph-mon[34941]: pgmap v440: 129 pgs: 129 active+clean; 965 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 13 MiB/s wr, 1.41k op/s 2024-09-05T01:18:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 75.0 s, 1057.2 tps, lat 94.591 ms stddev 90.413 2024-09-05T01:18:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.068+0000 7face8ff9640 1 -- 172.21.15.129:0/3777353886 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0cb909d70 con 0x55a0cba200f0 2024-09-05T01:18:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.068+0000 7face8ff9640 10 client.25177 handle_command_reply: tid=0 2024-09-05T01:18:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.068+0000 7fad0a0b9740 1 client.25177 shutdown 2024-09-05T01:18:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 msgr2=0x55a0cb9add20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7facfc005fd0 tx=0x7facfc014000 comp rx=0 tx=0).stop 2024-09-05T01:18:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 shutdown_connections 2024-09-05T01:18:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a0cba200f0 0x55a0cba22530 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a0cb9a4370 0x55a0cb9af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0cb99f2e0 0x55a0cb9ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 --2- 172.21.15.129:0/3777353886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a0cb99e9a0 0x55a0cb9add20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 >> 172.21.15.129:0/3777353886 conn(0x55a0cb99a010 msgr2=0x55a0cb9991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.069+0000 7face8ff9640 0 client.25177 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.070+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 shutdown_connections 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.070+0000 7fad0a0b9740 1 -- 172.21.15.129:0/3777353886 wait complete. 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.070+0000 7fad0a0b9740 20 client.25177 trim_cache size 0 max 16384 2024-09-05T01:18:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.071+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 msgr2=0x7facec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.071+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 0x7facec0a52e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55a0cb99ae90 tx=0x7facfc005a20 comp rx=0 tx=0).stop 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.071+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 msgr2=0x7fad00196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.071+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 0x7fad00196060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf002a860 tx=0x7facf002ced0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 shutdown_connections 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7facec0a2e20 0x7facec0a52e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad00105930 0x7fad0019d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fad00104f70 0x7fad00196060 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 --2- 172.21.15.129:0/297392824 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fad00103d80 0x7fad00195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 >> 172.21.15.129:0/297392824 conn(0x7fad000ff4e0 msgr2=0x7fad00078790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 shutdown_connections 2024-09-05T01:18:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:21.072+0000 7fad07326640 1 -- 172.21.15.129:0/297392824 wait complete. 2024-09-05T01:18:21.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:20 smithi183 ceph-mon[34589]: pgmap v441: 129 pgs: 129 active+clean; 969 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-05T01:18:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:20 smithi129 ceph-mon[30979]: pgmap v441: 129 pgs: 129 active+clean; 969 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-05T01:18:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:20 smithi194 ceph-mon[34941]: pgmap v441: 129 pgs: 129 active+clean; 969 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 MiB/s wr, 1.42k op/s 2024-09-05T01:18:23.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:22 smithi183 ceph-mon[34589]: pgmap v442: 129 pgs: 129 active+clean; 969 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-05T01:18:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:22 smithi129 ceph-mon[30979]: pgmap v442: 129 pgs: 129 active+clean; 969 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-05T01:18:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:22 smithi194 ceph-mon[34941]: pgmap v442: 129 pgs: 129 active+clean; 969 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.14k op/s 2024-09-05T01:18:25.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:24 smithi183 ceph-mon[34589]: pgmap v443: 129 pgs: 129 active+clean; 970 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-05T01:18:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 80.0 s, 1103.6 tps, lat 90.753 ms stddev 30.494 2024-09-05T01:18:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:24 smithi129 ceph-mon[30979]: pgmap v443: 129 pgs: 129 active+clean; 970 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-05T01:18:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:24 smithi194 ceph-mon[34941]: pgmap v443: 129 pgs: 129 active+clean; 970 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 14 MiB/s wr, 1.47k op/s 2024-09-05T01:18:27.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:26 smithi183 ceph-mon[34589]: pgmap v444: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-05T01:18:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:26 smithi129 ceph-mon[30979]: pgmap v444: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-05T01:18:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:26 smithi194 ceph-mon[34941]: pgmap v444: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 11 MiB/s wr, 1.17k op/s 2024-09-05T01:18:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:28 smithi129 ceph-mon[30979]: pgmap v445: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-05T01:18:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:28 smithi194 ceph-mon[34941]: pgmap v445: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-05T01:18:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:28 smithi183 ceph-mon[34589]: pgmap v445: 129 pgs: 129 active+clean; 970 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 12 MiB/s wr, 1.23k op/s 2024-09-05T01:18:30.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 85.0 s, 1183.8 tps, lat 84.481 ms stddev 22.130 2024-09-05T01:18:31.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:30 smithi183 ceph-mon[34589]: pgmap v446: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-05T01:18:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:30 smithi129 ceph-mon[30979]: pgmap v446: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-05T01:18:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:30 smithi194 ceph-mon[34941]: pgmap v446: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 MiB/s wr, 1.34k op/s 2024-09-05T01:18:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:32 smithi129 ceph-mon[30979]: pgmap v447: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:18:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:32 smithi194 ceph-mon[34941]: pgmap v447: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:18:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:32 smithi183 ceph-mon[34589]: pgmap v447: 129 pgs: 129 active+clean; 970 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:18:34.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.221+0000 7f06c359e640 1 -- 172.21.15.129:0/2333899110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4103050 msgr2=0x7f06c4103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.221+0000 7f06c359e640 1 --2- 172.21.15.129:0/2333899110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4103050 0x7f06c4103450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009ca0 tx=0x7f06bc02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.222+0000 7f06c359e640 1 -- 172.21.15.129:0/2333899110 shutdown_connections 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.222+0000 7f06c359e640 1 --2- 172.21.15.129:0/2333899110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.222+0000 7f06c359e640 1 --2- 172.21.15.129:0/2333899110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4101580 0x7f06c4101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.222+0000 7f06c359e640 1 --2- 172.21.15.129:0/2333899110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4103050 0x7f06c4103450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.222+0000 7f06c359e640 1 -- 172.21.15.129:0/2333899110 >> 172.21.15.129:0/2333899110 conn(0x7f06c40fd0d0 msgr2=0x7f06c40ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.223+0000 7f06c359e640 1 -- 172.21.15.129:0/2333899110 shutdown_connections 2024-09-05T01:18:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.223+0000 7f06c359e640 1 -- 172.21.15.129:0/2333899110 wait complete. 2024-09-05T01:18:34.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.224+0000 7f06c359e640 1 Processor -- start 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.224+0000 7f06c359e640 1 -- start start 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.224+0000 7f06c359e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4101580 0x7f06c4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c359e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 0x7f06c419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c359e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d650 con 0x7f06c4103050 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c359e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d4d0 con 0x7f06c4101f40 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c359e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06c410d7d0 con 0x7f06c4101580 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 0x7f06c419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 0x7f06c419f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51314/0 (socket says 172.21.15.129:51314) 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c2d9d640 1 -- 172.21.15.129:0/1536385683 learned_addr learned my addr 172.21.15.129:0/1536385683 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c1d9b640 1 -- 172.21.15.129:0/1536385683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4101580 msgr2=0x7f06c4197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4101580 0x7f06c4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c1d9b640 1 -- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 msgr2=0x7f06c419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.225+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 0x7f06c419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.226+0000 7f06c1d9b640 1 -- 172.21.15.129:0/1536385683 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c406bd70 con 0x7f06c4101f40 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.226+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4198260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06b800af30 tx=0x7f06b800cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.227+0000 7f06b37fe640 1 -- 172.21.15.129:0/1536385683 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b8007bb0 con 0x7f06c4101f40 2024-09-05T01:18:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.227+0000 7f06b37fe640 1 -- 172.21.15.129:0/1536385683 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06b8007d50 con 0x7f06c4101f40 2024-09-05T01:18:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.227+0000 7f06b37fe640 1 -- 172.21.15.129:0/1536385683 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b803a760 con 0x7f06c4101f40 2024-09-05T01:18:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.227+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c406c060 con 0x7f06c4101f40 2024-09-05T01:18:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.227+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c406c560 con 0x7f06c4101f40 2024-09-05T01:18:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.230+0000 7f06cbc9a740 1 Processor -- start 2024-09-05T01:18:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 -- start start 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621081994b0 0x56210819df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x5621081a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5621080fc2f0 con 0x56210819e540 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621080a4270 con 0x562108198b70 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5621080f82b0 con 0x5621081994b0 2024-09-05T01:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40662/0 (socket says 172.21.15.129:40662) 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.231+0000 7f06c2d9d640 1 -- 172.21.15.129:0/3090592066 learned_addr learned my addr 172.21.15.129:0/3090592066 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.232+0000 7f06c259c640 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621081994b0 0x56210819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06b37fe640 1 -- 172.21.15.129:0/1536385683 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f06b8031460 con 0x7f06c4101f40 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06c2d9d640 1 -- 172.21.15.129:0/3090592066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621081994b0 msgr2=0x56210819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621081994b0 0x56210819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06c2d9d640 1 -- 172.21.15.129:0/3090592066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 msgr2=0x5621081a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x5621081a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.233+0000 7f06c2d9d640 1 -- 172.21.15.129:0/3090592066 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621080a3340 con 0x562108198b70 2024-09-05T01:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.235+0000 7f06b37fe640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 0x7f06ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.235+0000 7f06b37fe640 1 -- 172.21.15.129:0/1536385683 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f06b80f15b0 con 0x7f06c4101f40 2024-09-05T01:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.235+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009aa0 tx=0x7f06b4009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f265f8703fa0337 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.236+0000 7f06b0ff9640 1 -- 172.21.15.129:0/3090592066 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4017070 con 0x562108198b70 2024-09-05T01:18:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.236+0000 7f06c259c640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 0x7f06ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.237+0000 7f06b0ff9640 1 -- 172.21.15.129:0/3090592066 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06b4013470 con 0x562108198b70 2024-09-05T01:18:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.237+0000 7f06b0ff9640 1 -- 172.21.15.129:0/3090592066 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4012610 con 0x562108198b70 2024-09-05T01:18:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.237+0000 7f06c259c640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 0x7f06ac0a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06bc02aeb0 tx=0x7f06bc031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 -- 172.21.15.129:0/3090592066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 msgr2=0x562108198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009aa0 tx=0x7f06b4009f60 comp rx=0 tx=0).stop 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 -- 172.21.15.129:0/3090592066 shutdown_connections 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x5621081a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5621081994b0 0x56210819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/3090592066 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x562108198f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.239+0000 7f06cbc9a740 1 -- 172.21.15.129:0/3090592066 >> 172.21.15.129:0/3090592066 conn(0x5621081941e0 msgr2=0x5621081945e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.240+0000 7f06cbc9a740 1 -- 172.21.15.129:0/3090592066 shutdown_connections 2024-09-05T01:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.240+0000 7f06cbc9a740 1 -- 172.21.15.129:0/3090592066 wait complete. 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.240+0000 7f06cbc9a740 1 -- start start 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.240+0000 7f06cbc9a740 1 Processor -- start 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562108198b70 0x5621081a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5621081a1510 con 0x562108198b70 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562108085290 con 0x5621081994b0 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5621081a1690 con 0x56210819e540 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06cbc9a740 10 client.0 _ll_register_callbacks cb 0x5621080b6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06c1d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40676/0 (socket says 172.21.15.129:40676) 2024-09-05T01:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06c1d9b640 1 -- 172.21.15.129:0/1007735839 learned_addr learned my addr 172.21.15.129:0/1007735839 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.241+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.242+0000 7f06c2d9d640 1 -- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 msgr2=0x5621081a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.242+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.242+0000 7f06c2d9d640 1 -- 172.21.15.129:0/1007735839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562108198b70 msgr2=0x5621081a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.242+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562108198b70 0x5621081a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.242+0000 7f06c2d9d640 1 -- 172.21.15.129:0/1007735839 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562107e0f210 con 0x56210819e540 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.243+0000 7f06c2d9d640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a9490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009aa0 tx=0x7f06b40097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d2396c622a8aae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.243+0000 7f06a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.243+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4017050 con 0x56210819e540 2024-09-05T01:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.243+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f06b401c450 con 0x56210819e540 2024-09-05T01:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.244+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b4013470 con 0x56210819e540 2024-09-05T01:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.244+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06b402f050 con 0x56210819e540 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.244+0000 7f06cbc9a740 10 client.25186 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.244+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562107edb620 con 0x56210819e540 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.245+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06b4013470 con 0x56210819e540 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f06b4026830 con 0x56210819e540 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 10 client.25186 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:34.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 10 client.25186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 20 client.25186 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5621081bf7d0 0x5621081c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 4 client.25186 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.246+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562108109a20 con 0x5621081bf7d0 2024-09-05T01:18:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.247+0000 7f06c259c640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5621081bf7d0 0x5621081c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.247+0000 7f06c259c640 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5621081bf7d0 0x5621081c1bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7b497a46405133 server_cookie=70efe3e966fa1a16 in_seq=0 out_seq=0 2024-09-05T01:18:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.248+0000 7f06a9ffb640 10 client.25186 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.248+0000 7f06a9ffb640 1 -- 172.21.15.129:0/1007735839 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562108109a20 con 0x5621081bf7d0 2024-09-05T01:18:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06a9ffb640 10 client.25186 handle_command_reply: tid=0 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 client.25186 shutdown 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 msgr2=0x5621081a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a9490 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009aa0 tx=0x7f06b40097a0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 shutdown_connections 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5621081bf7d0 0x5621081c1bc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a9490 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1007735839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562108198b70 0x5621081a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 >> 172.21.15.129:0/1007735839 conn(0x5621081941e0 msgr2=0x5621081959f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.249+0000 7f06a9ffb640 0 client.25186 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.250+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 shutdown_connections 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.250+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1007735839 wait complete. 2024-09-05T01:18:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.250+0000 7f06cbc9a740 20 client.25186 trim_cache size 0 max 16384 2024-09-05T01:18:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.282+0000 7f06cbc9a740 1 Processor -- start 2024-09-05T01:18:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.282+0000 7f06cbc9a740 1 -- start start 2024-09-05T01:18:34.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.282+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 0x5621081955d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.282+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.282+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x562108195b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41a2770 con 0x56210819e540 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41a0040 con 0x5621081994b0 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4074cf0 con 0x562108198b70 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x562108195b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x562108195b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51346/0 (socket says 172.21.15.129:51346) 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c2d9d640 1 -- 172.21.15.129:0/1629991439 learned_addr learned my addr 172.21.15.129:0/1629991439 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 0x5621081955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 -- 172.21.15.129:0/1629991439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 msgr2=0x5621081a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 -- 172.21.15.129:0/1629991439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 msgr2=0x562108195b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x562108195b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.283+0000 7f06c259c640 1 -- 172.21.15.129:0/1629991439 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621081bc240 con 0x562108198b70 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.284+0000 7f06c259c640 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 0x5621081955d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06bc031970 tx=0x7f06bc03f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=651b05610d91e0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.284+0000 7f06abfff640 1 -- 172.21.15.129:0/1629991439 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc031e10 con 0x562108198b70 2024-09-05T01:18:34.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.284+0000 7f06abfff640 1 -- 172.21.15.129:0/1629991439 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06bc036040 con 0x562108198b70 2024-09-05T01:18:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.284+0000 7f06abfff640 1 -- 172.21.15.129:0/1629991439 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc030cc0 con 0x562108198b70 2024-09-05T01:18:34.286 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1629991439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 msgr2=0x5621081955d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 0x5621081955d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f06bc031970 tx=0x7f06bc03f000 comp rx=0 tx=0).stop 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1629991439 shutdown_connections 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56210819e540 0x562108195b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5621081994b0 0x5621081a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/1629991439 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562108198b70 0x5621081955d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.285+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1629991439 >> 172.21.15.129:0/1629991439 conn(0x5621081941e0 msgr2=0x562108192a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.286+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1629991439 shutdown_connections 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.286+0000 7f06cbc9a740 1 -- 172.21.15.129:0/1629991439 wait complete. 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.286+0000 7f06cbc9a740 1 -- start start 2024-09-05T01:18:34.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.286+0000 7f06cbc9a740 1 Processor -- start 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x5621081a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 0x5621081a7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4075360 con 0x5621081994b0 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f06c419f970 con 0x562108198b70 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4075a30 con 0x56210819e540 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06cbc9a740 10 client.0 _ll_register_callbacks cb 0x7f06bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:34.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06c2d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:34940/0 (socket says 172.21.15.129:34940) 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06c2d9d640 1 -- 172.21.15.129:0/2472835142 learned_addr learned my addr 172.21.15.129:0/2472835142 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 0x5621081a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.287+0000 7f06c259c640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x5621081a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c259c640 1 -- 172.21.15.129:0/2472835142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 msgr2=0x5621081a8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c259c640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c259c640 1 -- 172.21.15.129:0/2472835142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 msgr2=0x5621081a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c259c640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 0x5621081a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c259c640 1 -- 172.21.15.129:0/2472835142 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621081a8840 con 0x562108198b70 2024-09-05T01:18:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.288+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 0x5621081a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:18:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.289+0000 7f06c259c640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x5621081a6ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0317e0 tx=0x7f06bc0095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=582b43fb204cba3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:34.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.290+0000 7f06a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:18:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc05c030 con 0x562108198b70 2024-09-05T01:18:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9266+0+0 (secure 0 0 0) 0x7f06bc055040 con 0x562108198b70 2024-09-05T01:18:34.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06bc036040 con 0x562108198b70 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06bc066050 con 0x562108198b70 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06cbc9a740 10 client.25232 fetch_fsmap learned FSMap version 28 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56210822a330 con 0x562108198b70 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.291+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f06bc036040 con 0x562108198b70 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f06bc043040 con 0x562108198b70 2024-09-05T01:18:34.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 10 client.25232 fetch_fsmap finished waiting for FSMap version 28 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 20 client.25232 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56210822ac40 0x56210822d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 4 client.25232 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562108103d70 con 0x56210822ac40 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.292+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56210822ac40 0x56210822d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:34.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.293+0000 7f06a9ffb640 10 client.25232 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:34.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.294+0000 7f06c1d9b640 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56210822ac40 0x56210822d060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add82dc05fe97aa8 server_cookie=2b84bd760152d60c in_seq=0 out_seq=0 2024-09-05T01:18:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.316+0000 7f06a9ffb640 1 -- 172.21.15.129:0/2472835142 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562108103d70 con 0x56210822ac40 2024-09-05T01:18:34.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.316+0000 7f06a9ffb640 10 client.25232 handle_command_reply: tid=0 2024-09-05T01:18:34.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.316+0000 7f06cbc9a740 1 client.25232 shutdown 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 msgr2=0x5621081a6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x5621081a6ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0317e0 tx=0x7f06bc0095d0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 shutdown_connections 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56210822ac40 0x56210822d060 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56210819e540 0x5621081a8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5621081994b0 0x5621081a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 --2- 172.21.15.129:0/2472835142 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562108198b70 0x5621081a6ce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.317+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 >> 172.21.15.129:0/2472835142 conn(0x5621081941e0 msgr2=0x5621081932b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.318+0000 7f06a9ffb640 0 client.25232 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.318+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 shutdown_connections 2024-09-05T01:18:34.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.318+0000 7f06cbc9a740 1 -- 172.21.15.129:0/2472835142 wait complete. 2024-09-05T01:18:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.318+0000 7f06cbc9a740 20 client.25232 trim_cache size 0 max 16384 2024-09-05T01:18:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:18:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:18:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:18:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.319+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 msgr2=0x7f06ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.319+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 0x7f06ac0a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f06bc02aeb0 tx=0x7f06bc031000 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.319+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 msgr2=0x7f06c4198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.319+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4198260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f06b800af30 tx=0x7f06b800cdd0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 shutdown_connections 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f06ac0a2b80 0x7f06ac0a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f06c4103050 0x7f06c419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f06c4101f40 0x7f06c4198260 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 --2- 172.21.15.129:0/1536385683 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f06c4101580 0x7f06c4197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 >> 172.21.15.129:0/1536385683 conn(0x7f06c40fd0d0 msgr2=0x7f06c4105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.320+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 shutdown_connections 2024-09-05T01:18:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:34.321+0000 7f06c359e640 1 -- 172.21.15.129:0/1536385683 wait complete. 2024-09-05T01:18:34.925 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T01:18:34.926 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T01:18:34.926 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:18:35.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 90.0 s, 1130.2 tps, lat 88.065 ms stddev 24.803 2024-09-05T01:18:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:34 smithi129 ceph-mon[30979]: pgmap v448: 129 pgs: 129 active+clean; 970 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-05T01:18:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:34 smithi129 ceph-mon[30979]: daemon mds.c restarted 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-mon[34941]: pgmap v448: 129 pgs: 129 active+clean; 970 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-mon[34941]: daemon mds.c restarted 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setuser ceph since I am not root 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setgroup ceph since I am not root 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-mds[65377]: -- 172.21.15.194:0/4167545245 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557cf7ce7860 con 0x557cf7cc5000 2024-09-05T01:18:35.250 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:18:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: starting mds.c at 2024-09-05T01:18:35.365 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:18:35.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:34 smithi183 ceph-mon[34589]: pgmap v448: 129 pgs: 129 active+clean; 970 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 1.47k op/s 2024-09-05T01:18:35.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:34 smithi183 ceph-mon[34589]: daemon mds.c restarted 2024-09-05T01:18:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:35 smithi129 ceph-mon[30979]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T01:18:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:35 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6836/2216480546,v1:172.21.15.194:6837/2216480546] up:boot 2024-09-05T01:18:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:35 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:18:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:35 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:18:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:35 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:18:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:35 smithi194 ceph-mon[34941]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T01:18:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:35 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6836/2216480546,v1:172.21.15.194:6837/2216480546] up:boot 2024-09-05T01:18:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:35 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:18:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:35 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:18:36.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:35 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:18:36.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:35 smithi183 ceph-mon[34589]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T01:18:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:35 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6836/2216480546,v1:172.21.15.194:6837/2216480546] up:boot 2024-09-05T01:18:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:35 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:18:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:35 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:18:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:35 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:18:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:36 smithi129 ceph-mon[30979]: pgmap v450: 129 pgs: 129 active+clean; 970 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-09-05T01:18:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:36 smithi194 ceph-mon[34941]: pgmap v450: 129 pgs: 129 active+clean; 970 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-09-05T01:18:37.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:36 smithi183 ceph-mon[34589]: pgmap v450: 129 pgs: 129 active+clean; 970 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 11 MiB/s wr, 1.27k op/s 2024-09-05T01:18:37.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.759+0000 7f351f7fe640 1 -- 172.21.15.129:0/2229821544 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510031e50 con 0x7f35201011e0 2024-09-05T01:18:37.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.760+0000 7f35270a4640 1 -- 172.21.15.129:0/2229821544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35201011e0 msgr2=0x7f3520105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:37.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.760+0000 7f35270a4640 1 --2- 172.21.15.129:0/2229821544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35201011e0 0x7f3520105980 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3510009cd0 tx=0x7f351002a930 comp rx=0 tx=0).stop 2024-09-05T01:18:37.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 -- 172.21.15.129:0/2229821544 shutdown_connections 2024-09-05T01:18:37.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 --2- 172.21.15.129:0/2229821544 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3520105ec0 0x7f3520106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 --2- 172.21.15.129:0/2229821544 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35201011e0 0x7f3520105980 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 --2- 172.21.15.129:0/2229821544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35201008c0 0x7f3520100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 -- 172.21.15.129:0/2229821544 >> 172.21.15.129:0/2229821544 conn(0x7f35200fc7d0 msgr2=0x7f35200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 -- 172.21.15.129:0/2229821544 shutdown_connections 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.761+0000 7f35270a4640 1 -- 172.21.15.129:0/2229821544 wait complete. 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.762+0000 7f35270a4640 1 Processor -- start 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.762+0000 7f35270a4640 1 -- start start 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.762+0000 7f35270a4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35201008c0 0x7f3520073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:37.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.762+0000 7f35270a4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.762+0000 7f35270a4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3520105ec0 0x7f352006d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f35270a4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35201185b0 con 0x7f35201008c0 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f35270a4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3520118430 con 0x7f35201011e0 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f35270a4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3520118730 con 0x7f3520105ec0 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:37.767 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40712/0 (socket says 172.21.15.129:40712) 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 -- 172.21.15.129:0/3135919718 learned_addr learned my addr 172.21.15.129:0/3135919718 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f3525641640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3520105ec0 0x7f352006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 -- 172.21.15.129:0/3135919718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3520105ec0 msgr2=0x7f352006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3520105ec0 0x7f352006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 -- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35201008c0 msgr2=0x7f3520073ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:18:37.768 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35201008c0 0x7f3520073ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:37.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 -- 172.21.15.129:0/3135919718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352006d730 con 0x7f35201011e0 2024-09-05T01:18:37.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.763+0000 7f351ffff640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3510031fd0 tx=0x7f3510009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:37.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.764+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3510030b00 con 0x7f35201011e0 2024-09-05T01:18:37.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.764+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f351002d070 con 0x7f35201011e0 2024-09-05T01:18:37.769 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.764+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351003b660 con 0x7f35201011e0 2024-09-05T01:18:37.770 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.764+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35200715f0 con 0x7f35201011e0 2024-09-05T01:18:37.771 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.764+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3520071ae0 con 0x7f35201011e0 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.766+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352006a850 con 0x7f35201011e0 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.768+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3510030ca0 con 0x7f35201011e0 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.769+0000 7f351dffb640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 0x7f35080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.770+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f35100f0870 con 0x7f35201011e0 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.770+0000 7f3524e40640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 0x7f35080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:37.772 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.771+0000 7f3524e40640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 0x7f35080a5210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3518009a60 tx=0x7f3518005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:37.774 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:37.774+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35100453e0 con 0x7f35201011e0 2024-09-05T01:18:38.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.168+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3520105980 con 0x7f35201011e0 2024-09-05T01:18:38.171 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:18:38.171 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:18:38.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.169+0000 7f351dffb640 1 -- 172.21.15.129:0/3135919718 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f3520105980 con 0x7f35201011e0 2024-09-05T01:18:38.171 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 msgr2=0x7f35080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 0x7f35080a5210 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3518009a60 tx=0x7f3518005f90 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 msgr2=0x7f352006cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3510031fd0 tx=0x7f3510009780 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 shutdown_connections 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f35080a2d50 0x7f35080a5210 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3520105ec0 0x7f352006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35201011e0 0x7f352006cb50 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3510031fd0 tx=0x7f3510009780 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 --2- 172.21.15.129:0/3135919718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35201008c0 0x7f3520073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:38.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.173+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 >> 172.21.15.129:0/3135919718 conn(0x7f35200fc7d0 msgr2=0x7f352010f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:38.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.174+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 shutdown_connections 2024-09-05T01:18:38.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:38.174+0000 7f35270a4640 1 -- 172.21.15.129:0/3135919718 wait complete. 2024-09-05T01:18:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:38 smithi129 ceph-mon[30979]: pgmap v451: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-09-05T01:18:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:38 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3135919718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:38 smithi194 ceph-mon[34941]: pgmap v451: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-09-05T01:18:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:38 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3135919718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:39.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:38 smithi183 ceph-mon[34589]: pgmap v451: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.29k op/s 2024-09-05T01:18:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:38 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3135919718' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 95.0 s, 912.2 tps, lat 110.158 ms stddev 47.672 2024-09-05T01:18:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:40 smithi129 ceph-mon[30979]: pgmap v452: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-05T01:18:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:40 smithi194 ceph-mon[34941]: pgmap v452: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-05T01:18:41.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:40 smithi183 ceph-mon[34589]: pgmap v452: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 10 MiB/s wr, 1.24k op/s 2024-09-05T01:18:41.430 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T01:18:41.430 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T01:18:41.895 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:18:42.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:42 smithi129 ceph-mon[30979]: pgmap v453: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-09-05T01:18:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 -- 172.21.15.129:0/4212439184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c411b140 msgr2=0x7fe0c411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/4212439184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c411b140 0x7fe0c411d530 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe0b401a910 tx=0x7fe0b403b450 comp rx=0 tx=0).stop 2024-09-05T01:18:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 -- 172.21.15.129:0/4212439184 shutdown_connections 2024-09-05T01:18:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/4212439184 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c411b140 0x7fe0c411d530 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/4212439184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/4212439184 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c4109450 0x7fe0c4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.153+0000 7fe0cabcf640 1 -- 172.21.15.129:0/4212439184 >> 172.21.15.129:0/4212439184 conn(0x7fe0c406e140 msgr2=0x7fe0c406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.155+0000 7fe0cabcf640 1 -- 172.21.15.129:0/4212439184 shutdown_connections 2024-09-05T01:18:43.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.155+0000 7fe0cabcf640 1 -- 172.21.15.129:0/4212439184 wait complete. 2024-09-05T01:18:43.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.155+0000 7fe0cabcf640 1 Processor -- start 2024-09-05T01:18:43.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 -- start start 2024-09-05T01:18:43.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c4109450 0x7fe0c411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41131a0 con 0x7fe0c4109830 2024-09-05T01:18:43.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4113020 con 0x7fe0c4109450 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0cabcf640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4113320 con 0x7fe0c411b140 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c916c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4116780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47682/0 (socket says 172.21.15.129:47682) 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c916c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38682/0 (socket says 172.21.15.129:38682) 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c916c640 1 -- 172.21.15.129:0/3883638890 learned_addr learned my addr 172.21.15.129:0/3883638890 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.156+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c4109450 0x7fe0c411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 -- 172.21.15.129:0/3883638890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c4109450 msgr2=0x7fe0c411a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c4109450 0x7fe0c411a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 -- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 msgr2=0x7fe0c4116780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4116780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 -- 172.21.15.129:0/3883638890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4117200 con 0x7fe0c411b140 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.157+0000 7fe0c916c640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4004ff0 tx=0x7fe0b4004a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.158+0000 7fe0c1ffb640 1 -- 172.21.15.129:0/3883638890 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b404f070 con 0x7fe0c411b140 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.158+0000 7fe0c1ffb640 1 -- 172.21.15.129:0/3883638890 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0b4004c10 con 0x7fe0c411b140 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.158+0000 7fe0c1ffb640 1 -- 172.21.15.129:0/3883638890 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4046030 con 0x7fe0c411b140 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.158+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c4117490 con 0x7fe0c411b140 2024-09-05T01:18:43.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.158+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41ab340 con 0x7fe0c411b140 2024-09-05T01:18:43.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.162+0000 7fe0ccfa8740 1 Processor -- start 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0c1ffb640 1 -- 172.21.15.129:0/3883638890 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe0b4052070 con 0x7fe0c411b140 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 -- start start 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c02794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5566c01d3160 con 0x5566c026f170 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566c017a720 con 0x5566c0279a30 2024-09-05T01:18:43.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5566c01cf120 con 0x5566c026e850 2024-09-05T01:18:43.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.163+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38688/0 (socket says 172.21.15.129:38688) 2024-09-05T01:18:43.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 -- 172.21.15.129:0/1081641284 learned_addr learned my addr 172.21.15.129:0/1081641284 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:43.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c896b640 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c02794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 -- 172.21.15.129:0/1081641284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 msgr2=0x5566c027be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 -- 172.21.15.129:0/1081641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 msgr2=0x5566c02794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c02794f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.164+0000 7fe0c3fff640 1 -- 172.21.15.129:0/1081641284 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566bff44ba0 con 0x5566c026e850 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0099a0 tx=0x7fe0bc009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=440a092a4d987087 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0c1ffb640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 0x7fe0ac0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0aaffd640 1 -- 172.21.15.129:0/1081641284 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc004b20 con 0x5566c026e850 2024-09-05T01:18:43.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0aaffd640 1 -- 172.21.15.129:0/1081641284 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0bc010d40 con 0x5566c026e850 2024-09-05T01:18:43.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0c1ffb640 1 -- 172.21.15.129:0/3883638890 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe0b41006b0 con 0x7fe0c411b140 2024-09-05T01:18:43.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.165+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 0x7fe0ac0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0aaffd640 1 -- 172.21.15.129:0/1081641284 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc005510 con 0x5566c026e850 2024-09-05T01:18:43.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/1081641284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 msgr2=0x5566c026ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0099a0 tx=0x7fe0bc009a70 comp rx=0 tx=0).stop 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/1081641284 shutdown_connections 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c02794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/1081641284 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c026ec30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/1081641284 >> 172.21.15.129:0/1081641284 conn(0x5566c026ade0 msgr2=0x5566c026b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.166+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 0x7fe0ac0a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5566c026bc40 tx=0x7fe0b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.167+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/1081641284 shutdown_connections 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.167+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/1081641284 wait complete. 2024-09-05T01:18:43.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.167+0000 7fe0ccfa8740 1 -- start start 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.168+0000 7fe0ccfa8740 1 Processor -- start 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.168+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026e850 0x5566c0276990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.168+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c0277410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5566c027c650 con 0x5566c026e850 2024-09-05T01:18:43.169 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5566c015b900 con 0x5566c0279a30 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5566c027c7d0 con 0x5566c026f170 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0ccfa8740 10 client.0 _ll_register_callbacks cb 0x5566c0159780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38694/0 (socket says 172.21.15.129:38694) 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.169+0000 7fe0c3fff640 1 -- 172.21.15.129:0/2878092422 learned_addr learned my addr 172.21.15.129:0/2878092422 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:43.170 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 -- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 msgr2=0x5566c0277410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:18:43.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c0277410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 -- 172.21.15.129:0/2878092422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026e850 msgr2=0x5566c0276990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026e850 0x5566c0276990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 -- 172.21.15.129:0/2878092422 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c019bc50 con 0x5566c026f170 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.170+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc004c50 tx=0x7fe0bc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=499479233ee93f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc005000 con 0x5566c026f170 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc010d40 con 0x5566c026f170 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe0bc0052f0 con 0x5566c026f170 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc0054f0 con 0x5566c026f170 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0ccfa8740 10 client.25213 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:43.172 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.171+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566c0288400 con 0x5566c026f170 2024-09-05T01:18:43.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0bc010d40 con 0x5566c026f170 2024-09-05T01:18:43.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe0bc01fad0 con 0x5566c026f170 2024-09-05T01:18:43.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 10 client.25213 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:43.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 10 client.25213 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 20 client.25213 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c0289b10 0x5566c028bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 4 client.25213 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.172+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566c01e0890 con 0x5566c0289b10 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.173+0000 7fe0c896b640 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c0289b10 0x5566c028bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.174+0000 7fe0c896b640 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c0289b10 0x5566c028bfb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7660f5edd7a09c server_cookie=83c38e7f28748069 in_seq=0 out_seq=0 2024-09-05T01:18:43.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.174+0000 7fe0a3fff640 10 client.25213 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:43.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.174+0000 7fe0a3fff640 1 -- 172.21.15.129:0/2878092422 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566c01e0890 con 0x5566c0289b10 2024-09-05T01:18:43.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.174+0000 7fe0a3fff640 10 client.25213 handle_command_reply: tid=0 2024-09-05T01:18:43.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.175+0000 7fe0ccfa8740 1 client.25213 shutdown 2024-09-05T01:18:43.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.175+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 msgr2=0x5566c0276ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.175+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc004c50 tx=0x7fe0bc004c80 comp rx=0 tx=0).stop 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.175+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 shutdown_connections 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.175+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c0289b10 0x5566c028bfb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c0277410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0276ed0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/2878092422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026e850 0x5566c0276990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 >> 172.21.15.129:0/2878092422 conn(0x5566c026ade0 msgr2=0x5566c0270b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0a3fff640 0 client.25213 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 shutdown_connections 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/2878092422 wait complete. 2024-09-05T01:18:43.177 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.176+0000 7fe0ccfa8740 20 client.25213 trim_cache size 0 max 16384 2024-09-05T01:18:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.208+0000 7fe0ccfa8740 1 Processor -- start 2024-09-05T01:18:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.208+0000 7fe0ccfa8740 1 -- start start 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.208+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c026e850 0x5566c02712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.208+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 0x5566c02918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b7c20 con 0x5566c0279a30 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b54f0 con 0x5566c026e850 2024-09-05T01:18:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0ccfa8740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b3340 con 0x5566c026f170 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38708/0 (socket says 172.21.15.129:38708) 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0c3fff640 1 -- 172.21.15.129:0/545286964 learned_addr learned my addr 172.21.15.129:0/545286964 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0c916c640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 0x5566c02918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.209+0000 7fe0c896b640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c026e850 0x5566c02712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 -- 172.21.15.129:0/545286964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c026e850 msgr2=0x5566c02712b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c026e850 0x5566c02712b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 -- 172.21.15.129:0/545286964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 msgr2=0x5566c02918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 0x5566c02918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 -- 172.21.15.129:0/545286964 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566c0272350 con 0x5566c026f170 2024-09-05T01:18:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c916c640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 0x5566c02918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:18:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.210+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc010860 tx=0x7fe0bc009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de22921ae6c58126 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.211+0000 7fe0aa7fc640 1 -- 172.21.15.129:0/545286964 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0285e0 con 0x5566c026f170 2024-09-05T01:18:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.211+0000 7fe0aa7fc640 1 -- 172.21.15.129:0/545286964 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0bc009d80 con 0x5566c026f170 2024-09-05T01:18:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.211+0000 7fe0aa7fc640 1 -- 172.21.15.129:0/545286964 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc005210 con 0x5566c026f170 2024-09-05T01:18:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/545286964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 msgr2=0x5566c0275260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc010860 tx=0x7fe0bc009960 comp rx=0 tx=0).stop 2024-09-05T01:18:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/545286964 shutdown_connections 2024-09-05T01:18:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c0279a30 0x5566c02918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026f170 0x5566c0275260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/545286964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c026e850 0x5566c02712b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.212+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/545286964 >> 172.21.15.129:0/545286964 conn(0x5566c026ade0 msgr2=0x5566c0278a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.213+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/545286964 shutdown_connections 2024-09-05T01:18:43.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.213+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/545286964 wait complete. 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.213+0000 7fe0ccfa8740 1 -- start start 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.213+0000 7fe0ccfa8740 1 Processor -- start 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.214+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.214+0000 7fe0ccfa8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c027d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.214+0000 7fe0c896b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.214+0000 7fe0c896b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38716/0 (socket says 172.21.15.129:38716) 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 -- 172.21.15.129:0/3935766802 learned_addr learned my addr 172.21.15.129:0/3935766802 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b39b0 con 0x5566c026f170 2024-09-05T01:18:43.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c916c640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b4e20 con 0x5566c0279a30 2024-09-05T01:18:43.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b4080 con 0x5566c026e850 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c027d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 -- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 msgr2=0x5566c027d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 -- 172.21.15.129:0/3935766802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 msgr2=0x5566c027d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c027d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.215+0000 7fe0c896b640 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566c0275940 con 0x5566c026e850 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c027d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0c896b640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5566c028c860 tx=0x7fe0b80090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c91071378bf322 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0c916c640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0a3fff640 1 -- 172.21.15.129:0/3935766802 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b802f020 con 0x5566c026e850 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.216+0000 7fe0a3fff640 1 -- 172.21.15.129:0/3935766802 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe0b8023c30 con 0x5566c026e850 2024-09-05T01:18:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0a3fff640 10 client.0 inactive, discarding osd_map(91..91 src has 1..91) 2024-09-05T01:18:43.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0ccfa8740 10 client.0 _ll_register_callbacks cb 0x7fe0bc0125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:43.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc009d80 con 0x5566c026e850 2024-09-05T01:18:43.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0a3fff640 1 -- 172.21.15.129:0/3935766802 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe0b803a050 con 0x5566c026e850 2024-09-05T01:18:43.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0ccfa8740 10 client.25219 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.217+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5566c026efa0 con 0x5566c026e850 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0a3fff640 1 -- 172.21.15.129:0/3935766802 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe0b802b440 con 0x5566c026e850 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 10 client.25219 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 10 client.25219 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 20 client.25219 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c02f49b0 0x5566c02f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 4 client.25219 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x7fe0bc009d80 con 0x5566c02f49b0 2024-09-05T01:18:43.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.218+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c02f49b0 0x5566c02f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:43.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.219+0000 7fe0c3fff640 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c02f49b0 0x5566c02f6e70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=327bb99367bad638 server_cookie=6b5b393bb6a9b410 in_seq=0 out_seq=0 2024-09-05T01:18:43.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.219+0000 7fe0a3fff640 10 client.25219 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:42 smithi194 ceph-mon[34941]: pgmap v453: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-09-05T01:18:43.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:42 smithi183 ceph-mon[34589]: pgmap v453: 129 pgs: 129 active+clean; 966 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 MiB/s wr, 1.22k op/s 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0a3fff640 1 -- 172.21.15.129:0/3935766802 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fe0bc009d80 con 0x5566c02f49b0 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0a3fff640 10 client.25219 handle_command_reply: tid=0 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 client.25219 shutdown 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 msgr2=0x5566c0282120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x5566c028c860 tx=0x7fe0b80090e0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 shutdown_connections 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5566c02f49b0 0x5566c02f6e70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5566c0279a30 0x5566c027d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5566c026f170 0x5566c027d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 --2- 172.21.15.129:0/3935766802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5566c026e850 0x5566c0282120 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 >> 172.21.15.129:0/3935766802 conn(0x5566c026ade0 msgr2=0x5566c0278a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0a3fff640 0 client.25219 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 shutdown_connections 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 1 -- 172.21.15.129:0/3935766802 wait complete. 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.486+0000 7fe0ccfa8740 20 client.25219 trim_cache size 0 max 16384 2024-09-05T01:18:43.488 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 10, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "bd64db62": { 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 10, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 141.8, 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:18:43.489 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:18:43.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 msgr2=0x7fe0ac0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 0x7fe0ac0a5250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5566c026bc40 tx=0x7fe0b8008040 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 msgr2=0x7fe0c4116cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4004ff0 tx=0x7fe0b4004a30 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 shutdown_connections 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0ac0a2d90 0x7fe0ac0a5250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0c411b140 0x7fe0c4116cc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0c4109830 0x7fe0c4116780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 --2- 172.21.15.129:0/3883638890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0c4109450 0x7fe0c411a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 >> 172.21.15.129:0/3883638890 conn(0x7fe0c406e140 msgr2=0x7fe0c4070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 shutdown_connections 2024-09-05T01:18:43.491 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:43.487+0000 7fe0cabcf640 1 -- 172.21.15.129:0/3883638890 wait complete. 2024-09-05T01:18:44.375 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bd64db62', quiesce took 9.4 seconds. Will release after: 12.35 2024-09-05T01:18:45.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 100.0 s, 611.1 tps, lat 99.142 ms stddev 36.158 2024-09-05T01:18:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:44 smithi129 ceph-mon[30979]: pgmap v454: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-09-05T01:18:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:44 smithi194 ceph-mon[34941]: pgmap v454: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-09-05T01:18:45.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:44 smithi183 ceph-mon[34589]: pgmap v454: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.9 MiB/s wr, 1.07k op/s 2024-09-05T01:18:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:46 smithi129 ceph-mon[30979]: pgmap v455: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-05T01:18:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:46 smithi194 ceph-mon[34941]: pgmap v455: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-05T01:18:47.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:46 smithi183 ceph-mon[34589]: pgmap v455: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.5 MiB/s wr, 1.02k op/s 2024-09-05T01:18:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.426+0000 7feda56dd640 1 -- 172.21.15.129:0/2982841645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda0100220 msgr2=0x7feda0107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.426+0000 7feda56dd640 1 --2- 172.21.15.129:0/2982841645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda0100220 0x7feda0107240 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fed94009900 tx=0x7fed9402a410 comp rx=0 tx=0).stop 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 -- 172.21.15.129:0/2982841645 shutdown_connections 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 --2- 172.21.15.129:0/2982841645 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda01077d0 0x7feda0109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 --2- 172.21.15.129:0/2982841645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda0100220 0x7feda0107240 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 --2- 172.21.15.129:0/2982841645 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 0x7feda00ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 -- 172.21.15.129:0/2982841645 >> 172.21.15.129:0/2982841645 conn(0x7feda00fb500 msgr2=0x7feda00fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 -- 172.21.15.129:0/2982841645 shutdown_connections 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 -- 172.21.15.129:0/2982841645 wait complete. 2024-09-05T01:18:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 Processor -- start 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 -- start start 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 0x7feda0102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.427+0000 7feda56dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0100220 0x7feda01011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feda010d6d0 con 0x7feda01077d0 2024-09-05T01:18:47.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda010d550 con 0x7feda0100220 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feda010d850 con 0x7feda00ff8e0 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47744/0 (socket says 172.21.15.129:47744) 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 -- 172.21.15.129:0/528965318 learned_addr learned my addr 172.21.15.129:0/528965318 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0100220 0x7feda01011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9effd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 0x7feda0102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 -- 172.21.15.129:0/528965318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 msgr2=0x7feda0102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 0x7feda0102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 -- 172.21.15.129:0/528965318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0100220 msgr2=0x7feda01011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0100220 0x7feda01011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 -- 172.21.15.129:0/528965318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0101fa0 con 0x7feda01077d0 2024-09-05T01:18:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fed9800eaa0 tx=0x7fed9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed87fff640 1 -- 172.21.15.129:0/528965318 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800cdb0 con 0x7feda01077d0 2024-09-05T01:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed87fff640 1 -- 172.21.15.129:0/528965318 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed98005020 con 0x7feda01077d0 2024-09-05T01:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda01a1e40 con 0x7feda01077d0 2024-09-05T01:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7fed87fff640 1 -- 172.21.15.129:0/528965318 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed980116d0 con 0x7feda01077d0 2024-09-05T01:18:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.428+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda01a2250 con 0x7feda01077d0 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed87fff640 1 -- 172.21.15.129:0/528965318 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fed980051d0 con 0x7feda01077d0 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 Processor -- start 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 -- start start 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cf620302f0 con 0x55cf620cd2e0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf61fd8270 con 0x55cf620d2370 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7feda8475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cf6202c2b0 con 0x55cf620cc9a0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed87fff640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 0x7fed880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47758/0 (socket says 172.21.15.129:47758) 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed9effd640 1 -- 172.21.15.129:0/1085839635 learned_addr learned my addr 172.21.15.129:0/1085839635 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.431+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed87fff640 1 -- 172.21.15.129:0/528965318 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed980cf620 con 0x7feda01077d0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 0x7fed880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 -- 172.21.15.129:0/1085839635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 msgr2=0x55cf620ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 -- 172.21.15.129:0/1085839635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 msgr2=0x55cf620d4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620d4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 -- 172.21.15.129:0/1085839635 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf61fd7340 con 0x55cf620cd2e0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9effd640 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fed90009c90 tx=0x7fed900117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80873e5e5c04708d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed9f7fe640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 0x7fed880a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed98098440 tx=0x7fed98086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed857fa640 1 -- 172.21.15.129:0/1085839635 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90012450 con 0x55cf620cd2e0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed857fa640 1 -- 172.21.15.129:0/1085839635 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed90012aa0 con 0x55cf620cd2e0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.432+0000 7fed857fa640 1 -- 172.21.15.129:0/1085839635 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9000fb50 con 0x55cf620cd2e0 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- 172.21.15.129:0/1085839635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 msgr2=0x55cf620d1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fed90009c90 tx=0x7fed900117b0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- 172.21.15.129:0/1085839635 shutdown_connections 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620d4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d1da0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 --2- 172.21.15.129:0/1085839635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- 172.21.15.129:0/1085839635 >> 172.21.15.129:0/1085839635 conn(0x55cf620c8010 msgr2=0x55cf620c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- 172.21.15.129:0/1085839635 shutdown_connections 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- 172.21.15.129:0/1085839635 wait complete. 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 -- start start 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.433+0000 7feda8475740 1 Processor -- start 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620d6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620de1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cf620d5340 con 0x55cf620cd2e0 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55cf61fb9290 con 0x55cf620d2370 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55cf620d54c0 con 0x55cf620cc9a0 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 10 client.0 _ll_register_callbacks cb 0x55cf61fea140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47770/0 (socket says 172.21.15.129:47770) 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 -- 172.21.15.129:0/3699427881 learned_addr learned my addr 172.21.15.129:0/3699427881 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620de1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 -- 172.21.15.129:0/3699427881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 msgr2=0x55cf620d6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620d6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620d6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 -- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 msgr2=0x55cf620de1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620de1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 -- 172.21.15.129:0/3699427881 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf61d43210 con 0x55cf620cd2e0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed9effd640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fed90009c60 tx=0x7fed9000fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51353ad5367f5a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90029030 con 0x55cf620cd2e0 2024-09-05T01:18:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed90020040 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.434+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed90012aa0 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed90033050 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 10 client.15699 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf61e0f620 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed90012aa0 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fed90004680 con 0x55cf620cd2e0 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 10 client.15699 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 20 client.15699 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf620e2610 0x55cf620e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 4 client.15699 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf6203da20 con 0x55cf620e2610 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.435+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf620e2610 0x55cf620e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.436+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf620e2610 0x55cf620e4a30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc02946a2ab53aec server_cookie=739a5e1168c5f1a in_seq=0 out_seq=0 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.436+0000 7fed82ff5640 10 client.15699 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.436+0000 7fed82ff5640 1 -- 172.21.15.129:0/3699427881 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf6203da20 con 0x55cf620e2610 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.436+0000 7fed82ff5640 10 client.15699 handle_command_reply: tid=0 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.436+0000 7feda8475740 1 client.15699 shutdown 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 msgr2=0x55cf620d7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fed90009c60 tx=0x7fed9000fac0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 shutdown_connections 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf620e2610 0x55cf620e4a30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620de1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620d7160 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7fed82ff5640 0 client.15699 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 --2- 172.21.15.129:0/3699427881 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620d6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 >> 172.21.15.129:0/3699427881 conn(0x55cf620c8010 msgr2=0x55cf620c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 shutdown_connections 2024-09-05T01:18:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 1 -- 172.21.15.129:0/3699427881 wait complete. 2024-09-05T01:18:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.437+0000 7feda8475740 20 client.15699 trim_cache size 0 max 16384 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 Processor -- start 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 -- start start 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 0x55cf620de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feda01af230 con 0x55cf620cc9a0 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda01acb00 con 0x55cf620d2370 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7feda8475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feda01aa950 con 0x55cf620cd2e0 2024-09-05T01:18:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7fed9e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7fed9e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47776/0 (socket says 172.21.15.129:47776) 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7fed9e7fc640 1 -- 172.21.15.129:0/1392934131 learned_addr learned my addr 172.21.15.129:0/1392934131 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.451+0000 7fed9effd640 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 0x55cf620de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 -- 172.21.15.129:0/1392934131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 msgr2=0x55cf620de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 0x55cf620de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 -- 172.21.15.129:0/1392934131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 msgr2=0x55cf620c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 -- 172.21.15.129:0/1392934131 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf620e5090 con 0x55cf620cc9a0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9effd640 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 0x55cf620de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed94000c00 tx=0x7fed94009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63966d5ac374240 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed84ff9640 1 -- 172.21.15.129:0/1392934131 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9404e020 con 0x55cf620cc9a0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed84ff9640 1 -- 172.21.15.129:0/1392934131 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed94003d60 con 0x55cf620cc9a0 2024-09-05T01:18:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7fed84ff9640 1 -- 172.21.15.129:0/1392934131 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94035040 con 0x55cf620cc9a0 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 -- 172.21.15.129:0/1392934131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 msgr2=0x55cf620c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fed94000c00 tx=0x7fed94009620 comp rx=0 tx=0).stop 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 -- 172.21.15.129:0/1392934131 shutdown_connections 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cd2e0 0x55cf620de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 --2- 172.21.15.129:0/1392934131 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cc9a0 0x55cf620c83f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.452+0000 7feda8475740 1 -- 172.21.15.129:0/1392934131 >> 172.21.15.129:0/1392934131 conn(0x55cf620c8010 msgr2=0x55cf620d6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- 172.21.15.129:0/1392934131 shutdown_connections 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- 172.21.15.129:0/1392934131 wait complete. 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- start start 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 Processor -- start 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feda01aafc0 con 0x55cf620cd2e0 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feda01ac430 con 0x55cf620d2370 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feda01ab690 con 0x55cf620cc9a0 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7feda8475740 10 client.0 _ll_register_callbacks cb 0x7fed94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47784/0 (socket says 172.21.15.129:47784) 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 -- 172.21.15.129:0/2780262043 learned_addr learned my addr 172.21.15.129:0/2780262043 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 -- 172.21.15.129:0/2780262043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 msgr2=0x55cf620db370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620db370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 -- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 msgr2=0x55cf620dc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9effd640 1 -- 172.21.15.129:0/2780262043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf620dced0 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.453+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed9effd640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed90009750 tx=0x7fed90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c830c200d17a936 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90029040 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed9000f440 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed94003d60 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed90012760 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 10 client.15705 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cf620de1f0 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed94003d60 con 0x55cf620cd2e0 2024-09-05T01:18:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fed900044d0 con 0x55cf620cd2e0 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 10 client.15705 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 20 client.15705 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf6214d7e0 0x55cf6214fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 4 client.15705 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf62037d70 con 0x55cf6214d7e0 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.454+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf6214d7e0 0x55cf6214fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.455+0000 7fed82ff5640 10 client.15705 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.455+0000 7fed9e7fc640 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf6214d7e0 0x55cf6214fc00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbf01c2bb1d67d48 server_cookie=84f938b6f8125dd7 in_seq=0 out_seq=0 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7fed82ff5640 1 -- 172.21.15.129:0/2780262043 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf62037d70 con 0x55cf6214d7e0 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7fed82ff5640 10 client.15705 handle_command_reply: tid=0 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 client.15705 shutdown 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 msgr2=0x55cf620db8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fed90009750 tx=0x7fed90009b50 comp rx=0 tx=0).stop 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 shutdown_connections 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55cf6214d7e0 0x55cf6214fc00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55cf620d2370 0x55cf620dc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cf620cd2e0 0x55cf620db8b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7fed82ff5640 0 client.15705 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 --2- 172.21.15.129:0/2780262043 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55cf620cc9a0 0x55cf620db370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 >> 172.21.15.129:0/2780262043 conn(0x55cf620c8010 msgr2=0x55cf620c69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 shutdown_connections 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 1 -- 172.21.15.129:0/2780262043 wait complete. 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.469+0000 7feda8475740 20 client.15705 trim_cache size 0 max 16384 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:18:47.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:18:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 msgr2=0x7fed880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 0x7fed880a50f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fed98098440 tx=0x7fed98086040 comp rx=0 tx=0).stop 2024-09-05T01:18:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 msgr2=0x7feda0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fed9800eaa0 tx=0x7fed9800ef60 comp rx=0 tx=0).stop 2024-09-05T01:18:47.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 shutdown_connections 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed880a2c30 0x7fed880a50f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feda01077d0 0x7feda0101710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feda0100220 0x7feda01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 --2- 172.21.15.129:0/528965318 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feda00ff8e0 0x7feda0102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 >> 172.21.15.129:0/528965318 conn(0x7feda00fb500 msgr2=0x7feda0103220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 shutdown_connections 2024-09-05T01:18:47.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:18:47.470+0000 7feda56dd640 1 -- 172.21.15.129:0/528965318 wait complete. 2024-09-05T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:48 smithi129 ceph-mon[30979]: pgmap v456: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 944 op/s 2024-09-05T01:18:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:48 smithi194 ceph-mon[34941]: pgmap v456: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 944 op/s 2024-09-05T01:18:49.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:48 smithi183 ceph-mon[34589]: pgmap v456: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 MiB/s wr, 944 op/s 2024-09-05T01:18:50.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 105.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:18:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:50 smithi129 ceph-mon[30979]: pgmap v457: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 750 op/s 2024-09-05T01:18:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:50 smithi194 ceph-mon[34941]: pgmap v457: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 750 op/s 2024-09-05T01:18:51.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:50 smithi183 ceph-mon[34589]: pgmap v457: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 MiB/s wr, 750 op/s 2024-09-05T01:18:52.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:52 smithi183 ceph-mon[34589]: pgmap v458: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-09-05T01:18:52.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:52 smithi129 ceph-mon[30979]: pgmap v458: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-09-05T01:18:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:52 smithi194 ceph-mon[34941]: pgmap v458: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 MiB/s wr, 514 op/s 2024-09-05T01:18:55.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:54 smithi183 ceph-mon[34589]: pgmap v459: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-09-05T01:18:55.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 110.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:18:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:54 smithi129 ceph-mon[30979]: pgmap v459: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-09-05T01:18:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:54 smithi194 ceph-mon[34941]: pgmap v459: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-09-05T01:18:56.726 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:18:56.991 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:18:57.112 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:56 smithi129 ceph-mon[30979]: pgmap v460: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-05T01:18:57.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:56 smithi183 ceph-mon[34589]: pgmap v460: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-05T01:18:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:56 smithi194 ceph-mon[34941]: pgmap v460: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-09-05T01:18:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- 172.21.15.129:0/4176749443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 msgr2=0x7f579410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 --2- 172.21.15.129:0/4176749443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579410e610 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f578c009ca0 tx=0x7f578c02a950 comp rx=0 tx=0).stop 2024-09-05T01:18:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- 172.21.15.129:0/4176749443 shutdown_connections 2024-09-05T01:18:57.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 --2- 172.21.15.129:0/4176749443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579410e610 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 --2- 172.21.15.129:0/4176749443 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 0x7f579410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 --2- 172.21.15.129:0/4176749443 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 0x7f5794103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- 172.21.15.129:0/4176749443 >> 172.21.15.129:0/4176749443 conn(0x7f57940fc7a0 msgr2=0x7f57940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- 172.21.15.129:0/4176749443 shutdown_connections 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- 172.21.15.129:0/4176749443 wait complete. 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 Processor -- start 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.482+0000 7f579aa32640 1 -- start start 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 0x7f5794198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 0x7f57941988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5794111e60 con 0x7f579410c170 2024-09-05T01:18:57.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5794111ce0 con 0x7f5794102d40 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f579aa32640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5794111fe0 con 0x7f5794103660 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53480/0 (socket says 172.21.15.129:53480) 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 -- 172.21.15.129:0/631965842 learned_addr learned my addr 172.21.15.129:0/631965842 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5793fff640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 0x7f5794198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f57937fe640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 0x7f57941988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 -- 172.21.15.129:0/631965842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 msgr2=0x7f57941988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 0x7f57941988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 -- 172.21.15.129:0/631965842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 msgr2=0x7f5794198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 0x7f5794198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 -- 172.21.15.129:0/631965842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579419d330 con 0x7f579410c170 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.483+0000 7f5798fcf640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c02f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:57.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.484+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c034040 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.484+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f578c003e00 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.484+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579419d5c0 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.484+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578c045480 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.484+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57941a4f00 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.485+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579406b070 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.486+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f578c037020 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.486+0000 7f57917fa640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 0x7f577c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.486+0000 7f5793fff640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 0x7f577c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.486+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f578c0f0520 con 0x7f579410c170 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.487+0000 7f5793fff640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 0x7f577c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57840058c0 tx=0x7f578400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:57.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.491+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f578c0bd100 con 0x7f579410c170 2024-09-05T01:18:57.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.687+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57941071e0 con 0x7f579410c170 2024-09-05T01:18:57.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.688+0000 7f57917fa640 1 -- 172.21.15.129:0/631965842 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f578c0bca00 con 0x7f579410c170 2024-09-05T01:18:57.689 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:18:57.689 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:18:57.689 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 msgr2=0x7f577c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 0x7f577c0a5040 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f57840058c0 tx=0x7f578400a2d0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 msgr2=0x7f579419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f578c006010 tx=0x7f578c02f640 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 shutdown_connections 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f577c0a2b80 0x7f577c0a5040 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f579410c170 0x7f579419cc50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5794103660 0x7f57941988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 --2- 172.21.15.129:0/631965842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5794102d40 0x7f5794198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 >> 172.21.15.129:0/631965842 conn(0x7f57940fc7a0 msgr2=0x7f57940fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 shutdown_connections 2024-09-05T01:18:57.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:57.690+0000 7f579aa32640 1 -- 172.21.15.129:0/631965842 wait complete. 2024-09-05T01:18:57.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:57 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/631965842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:58.160 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:57 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/631965842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:58.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:57 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/631965842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:18:58.451 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id bd64db62 --release --await' 2024-09-05T01:18:58.451 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id bd64db62 --release --await 2024-09-05T01:18:58.699 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:18:59.086 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:18:58 smithi129 ceph-mon[30979]: pgmap v461: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 717 KiB/s wr, 90 op/s 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 -- 172.21.15.129:0/4003498817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde810c180 msgr2=0x7efde810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 --2- 172.21.15.129:0/4003498817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde810c180 0x7efde810e620 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7efde0009ca0 tx=0x7efde002a950 comp rx=0 tx=0).stop 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 -- 172.21.15.129:0/4003498817 shutdown_connections 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 --2- 172.21.15.129:0/4003498817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde810c180 0x7efde810e620 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 --2- 172.21.15.129:0/4003498817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8103670 0x7efde810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 --2- 172.21.15.129:0/4003498817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde8102d50 0x7efde8103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.188 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 -- 172.21.15.129:0/4003498817 >> 172.21.15.129:0/4003498817 conn(0x7efde80fc7d0 msgr2=0x7efde80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 -- 172.21.15.129:0/4003498817 shutdown_connections 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.172+0000 7efdedea7640 1 -- 172.21.15.129:0/4003498817 wait complete. 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 Processor -- start 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 -- start start 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 0x7efde81009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 0x7efde8101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efde8111e30 con 0x7efde8103670 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efde8111cb0 con 0x7efde8102d50 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efdedea7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efde8111fb0 con 0x7efde810c180 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efde6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.189 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efde6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53506/0 (socket says 172.21.15.129:53506) 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.173+0000 7efde6ffd640 1 -- 172.21.15.129:0/3134156828 learned_addr learned my addr 172.21.15.129:0/3134156828 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde77fe640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 0x7efde81009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 -- 172.21.15.129:0/3134156828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 msgr2=0x7efde8101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde7fff640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 0x7efde8101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 0x7efde8101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 -- 172.21.15.129:0/3134156828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 msgr2=0x7efde81009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 0x7efde81009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 -- 172.21.15.129:0/3134156828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde81a6750 con 0x7efde8103670 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde7fff640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 0x7efde8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde77fe640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 0x7efde81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde6ffd640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c970 tx=0x7efddc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde4ff9640 1 -- 172.21.15.129:0/3134156828 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc007bb0 con 0x7efde8103670 2024-09-05T01:18:59.190 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde81a6a40 con 0x7efde8103670 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde4ff9640 1 -- 172.21.15.129:0/3134156828 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efddc007d50 con 0x7efde8103670 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efde4ff9640 1 -- 172.21.15.129:0/3134156828 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc005240 con 0x7efde8103670 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.174+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde81a6fd0 con 0x7efde8103670 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.176+0000 7efde4ff9640 1 -- 172.21.15.129:0/3134156828 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efddc0053e0 con 0x7efde8103670 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 Processor -- start 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 -- start start 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b4322440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641b4287620 con 0x5641b432d240 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641b422ee50 con 0x5641b4322060 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efdf0280740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5641b42835e0 con 0x5641b4322980 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde4ff9640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 0x7efdd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.191 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b4322440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36486/0 (socket says 172.21.15.129:36486) 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53520/0 (socket says 172.21.15.129:53520) 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 -- 172.21.15.129:0/2521738210 learned_addr learned my addr 172.21.15.129:0/2521738210 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde7fff640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 0x7efdd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde4ff9640 1 -- 172.21.15.129:0/3134156828 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7efddc0cf530 con 0x7efde8103670 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 -- 172.21.15.129:0/2521738210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 msgr2=0x5641b4322440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b4322440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 -- 172.21.15.129:0/2521738210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 msgr2=0x5641b432f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.192 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.177+0000 7efde77fe640 1 -- 172.21.15.129:0/2521738210 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b421fd40 con 0x5641b4322980 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.178+0000 7efde7fff640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 0x7efdd00a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efde0009a60 tx=0x7efde0005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.178+0000 7efde77fe640 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009c50 tx=0x7efdd800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d14139375d2557ad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.178+0000 7efdcdffb640 1 -- 172.21.15.129:0/2521738210 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8016070 con 0x5641b4322980 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.178+0000 7efdcdffb640 1 -- 172.21.15.129:0/2521738210 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdd80054d0 con 0x5641b4322980 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.178+0000 7efdcdffb640 1 -- 172.21.15.129:0/2521738210 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8011470 con 0x5641b4322980 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- 172.21.15.129:0/2521738210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 msgr2=0x5641b432cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009c50 tx=0x7efdd800e6c0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- 172.21.15.129:0/2521738210 shutdown_connections 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432cd00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 --2- 172.21.15.129:0/2521738210 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b4322440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- 172.21.15.129:0/2521738210 >> 172.21.15.129:0/2521738210 conn(0x5641b431e5f0 msgr2=0x5641b431e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- 172.21.15.129:0/2521738210 shutdown_connections 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- 172.21.15.129:0/2521738210 wait complete. 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 -- start start 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.179+0000 7efdf0280740 1 Processor -- start 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b432a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641b432fe10 con 0x5641b432d240 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641b420ff10 con 0x5641b4322060 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5641b432ff90 con 0x5641b4322980 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdf0280740 10 client.0 _ll_register_callbacks cb 0x5641b4233db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53530/0 (socket says 172.21.15.129:53530) 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 -- 172.21.15.129:0/696851002 learned_addr learned my addr 172.21.15.129:0/696851002 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde77fe640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b432a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde6ffd640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 -- 172.21.15.129:0/696851002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 msgr2=0x5641b432a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 -- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 msgr2=0x5641b432a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b432a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 -- 172.21.15.129:0/696851002 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b3f96580 con 0x5641b432d240 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efde7fff640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efde0003cd0 tx=0x7efde0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecd76b2ab246797 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.180+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde004f070 con 0x5641b432d240 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7efde0004650 con 0x5641b432d240 2024-09-05T01:18:59.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd80054d0 con 0x5641b432d240 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efde005d050 con 0x5641b432d240 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641b4060960 con 0x5641b432d240 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdd80054d0 con 0x5641b432d240 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efde0048040 con 0x5641b432d240 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 10 client.15723 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 20 client.15723 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b433cd00 0x5641b433f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 4 client.15723 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b4294b90 con 0x5641b433cd00 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.181+0000 7efde77fe640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b433cd00 0x5641b433f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.182+0000 7efdc6ffd640 10 client.15723 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.182+0000 7efde77fe640 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b433cd00 0x5641b433f150 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68eeb57efbe22797 server_cookie=b299e820624a4ae in_seq=0 out_seq=0 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.182+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696851002 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641b4294b90 con 0x5641b433cd00 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.182+0000 7efdc6ffd640 10 client.15723 handle_command_reply: tid=0 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 client.15723 shutdown 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 msgr2=0x5641b432aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efde0003cd0 tx=0x7efde0009360 comp rx=0 tx=0).stop 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 shutdown_connections 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b433cd00 0x5641b433f150 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b432aaf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b432a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 --2- 172.21.15.129:0/696851002 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b432a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.197 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 >> 172.21.15.129:0/696851002 conn(0x5641b431e5f0 msgr2=0x5641b43242d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdc6ffd640 0 client.15723 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:59.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 shutdown_connections 2024-09-05T01:18:59.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 1 -- 172.21.15.129:0/696851002 wait complete. 2024-09-05T01:18:59.198 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.183+0000 7efdf0280740 20 client.15723 trim_cache size 0 max 16384 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 Processor -- start 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 -- start start 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4344f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 0x5641b43272d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.200+0000 7efdf0280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efde81b37f0 con 0x5641b4322060 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efdf0280740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efde81b10c0 con 0x5641b432d240 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efdf0280740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efde81aef10 con 0x5641b4322980 2024-09-05T01:18:59.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53542/0 (socket says 172.21.15.129:53542) 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 -- 172.21.15.129:0/2812488982 learned_addr learned my addr 172.21.15.129:0/2812488982 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 -- 172.21.15.129:0/2812488982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 msgr2=0x5641b4344f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde6ffd640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4344f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4344f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde7fff640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 0x5641b43272d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 -- 172.21.15.129:0/2812488982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 msgr2=0x5641b43272d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 0x5641b43272d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 -- 172.21.15.129:0/2812488982 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b431e420 con 0x5641b4322060 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde7fff640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 0x5641b43272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efde77fe640 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5641b431f450 tx=0x7efdd8009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264d49311ecbe5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efdcd7fa640 1 -- 172.21.15.129:0/2812488982 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8016040 con 0x5641b4322060 2024-09-05T01:18:59.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efdcd7fa640 1 -- 172.21.15.129:0/2812488982 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdd8004680 con 0x5641b4322060 2024-09-05T01:18:59.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.201+0000 7efdcd7fa640 1 -- 172.21.15.129:0/2812488982 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8003b50 con 0x5641b4322060 2024-09-05T01:18:59.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- 172.21.15.129:0/2812488982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 msgr2=0x5641b4344a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x5641b431f450 tx=0x7efdd8009940 comp rx=0 tx=0).stop 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- 172.21.15.129:0/2812488982 shutdown_connections 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b432d240 0x5641b43272d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4344f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 --2- 172.21.15.129:0/2812488982 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b4322060 0x5641b4344a20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- 172.21.15.129:0/2812488982 >> 172.21.15.129:0/2812488982 conn(0x5641b431e5f0 msgr2=0x5641b4328940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- 172.21.15.129:0/2812488982 shutdown_connections 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- 172.21.15.129:0/2812488982 wait complete. 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.202+0000 7efdf0280740 1 -- start start 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 Processor -- start 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b43351f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4335730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efde81af580 con 0x5641b432d240 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efde81b09f0 con 0x5641b4322060 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efde81afc50 con 0x5641b4322980 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efdf0280740 10 client.0 _ll_register_callbacks cb 0x7efde0003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53550/0 (socket says 172.21.15.129:53550) 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 -- 172.21.15.129:0/696656264 learned_addr learned my addr 172.21.15.129:0/696656264 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde77fe640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b43351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde6ffd640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4335730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 -- 172.21.15.129:0/696656264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 msgr2=0x5641b4335730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4335730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 -- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 msgr2=0x5641b43351f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b43351f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde7fff640 1 -- 172.21.15.129:0/696656264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b43306e0 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.203+0000 7efde6ffd640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4335730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efde7fff640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efde0054e70 tx=0x7efde002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff3888f5b485100 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde004f070 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7efde0004650 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd8004680 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efde0068050 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 10 client.15729 fetch_fsmap learned FSMap version 30 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641b43a81e0 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdd8004680 con 0x5641b432d240 2024-09-05T01:18:59.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7efde0054460 con 0x5641b432d240 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 10 client.15729 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 10 client.15729 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 20 client.15729 populate_metadata read hostname 'smithi129' 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b43a8980 0x5641b43aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 4 client.15729 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bd64db62", "release": true, "await": true}] 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.204+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bd64db62", "release": true, "await": true}) -- 0x5641b428f0a0 con 0x5641b43a8980 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.205+0000 7efde77fe640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b43a8980 0x5641b43aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.205+0000 7efde77fe640 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b43a8980 0x5641b43aada0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7444b826461aae23 server_cookie=b47fc60e8f4b7207 in_seq=0 out_seq=0 2024-09-05T01:18:59.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.205+0000 7efdc6ffd640 10 client.15729 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:18:58 smithi194 ceph-mon[34941]: pgmap v461: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 717 KiB/s wr, 90 op/s 2024-09-05T01:18:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdc6ffd640 1 -- 172.21.15.129:0/696656264 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5641b428f0a0 con 0x5641b43a8980 2024-09-05T01:18:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdc6ffd640 10 client.15729 handle_command_reply: tid=0 2024-09-05T01:18:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 client.15729 shutdown 2024-09-05T01:18:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 msgr2=0x5641b43301a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efde0054e70 tx=0x7efde002f3a0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 shutdown_connections 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641b43a8980 0x5641b43aada0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641b432d240 0x5641b43301a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdc6ffd640 0 client.15729 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641b4322980 0x5641b4335730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 --2- 172.21.15.129:0/696656264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641b4322060 0x5641b43351f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 >> 172.21.15.129:0/696656264 conn(0x5641b431e5f0 msgr2=0x5641b432a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 shutdown_connections 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 1 -- 172.21.15.129:0/696656264 wait complete. 2024-09-05T01:18:59.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.261+0000 7efdf0280740 20 client.15729 trim_cache size 0 max 16384 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 12, 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "bd64db62": { 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 12, 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:18:59.264 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 141.8, 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:59.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:59.266 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:18:59.266 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:18:59.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.262+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 msgr2=0x7efdd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 0x7efdd00a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efde0009a60 tx=0x7efde0005f30 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 msgr2=0x7efde8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7efddc00c970 tx=0x7efddc00ce30 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 shutdown_connections 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efdd00a2b80 0x7efdd00a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efde810c180 0x7efde8101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efde8103670 0x7efde8100ef0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 --2- 172.21.15.129:0/3134156828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efde8102d50 0x7efde81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 >> 172.21.15.129:0/3134156828 conn(0x7efde80fc7d0 msgr2=0x7efde80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 shutdown_connections 2024-09-05T01:18:59.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:18:59.263+0000 7efdedea7640 1 -- 172.21.15.129:0/3134156828 wait complete. 2024-09-05T01:18:59.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:18:58 smithi183 ceph-mon[34589]: pgmap v461: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 717 KiB/s wr, 90 op/s 2024-09-05T01:19:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 115.0 s, 236.0 tps, lat 1438.205 ms stddev 4461.219 2024-09-05T01:19:00.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bd64db62', total seconds elapsed: 25.3 2024-09-05T01:19:00.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.75 2024-09-05T01:19:00.216 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:19:00.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 -- 172.21.15.129:0/1156380251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c109170 msgr2=0x7fe77c1095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 --2- 172.21.15.129:0/1156380251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c109170 0x7fe77c1095f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009860 tx=0x7fe76c02a340 comp rx=0 tx=0).stop 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 -- 172.21.15.129:0/1156380251 shutdown_connections 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 --2- 172.21.15.129:0/1156380251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109b30 0x7fe77c1103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 --2- 172.21.15.129:0/1156380251 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c109170 0x7fe77c1095f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 --2- 172.21.15.129:0/1156380251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c108410 0x7fe77c108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.623+0000 7fe77b59e640 1 -- 172.21.15.129:0/1156380251 >> 172.21.15.129:0/1156380251 conn(0x7fe77c0771d0 msgr2=0x7fe77c079610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.625+0000 7fe77b59e640 1 -- 172.21.15.129:0/1156380251 shutdown_connections 2024-09-05T01:19:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.626+0000 7fe77b59e640 1 -- 172.21.15.129:0/1156380251 wait complete. 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 Processor -- start 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 -- start start 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c108410 0x7fe77c080dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109170 0x7fe77c081310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c07b320 con 0x7fe77c109170 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c07b1a0 con 0x7fe77c109b30 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77b59e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe77c07b4a0 con 0x7fe77c108410 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58980/0 (socket says 172.21.15.129:58980) 2024-09-05T01:19:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe77ad9d640 1 -- 172.21.15.129:0/818670315 learned_addr learned my addr 172.21.15.129:0/818670315 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.627+0000 7fe779d9b640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109170 0x7fe77c081310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77a59c640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c108410 0x7fe77c080dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 -- 172.21.15.129:0/818670315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c108410 msgr2=0x7fe77c080dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c108410 0x7fe77c080dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 -- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109170 msgr2=0x7fe77c081310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109170 0x7fe77c081310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 -- 172.21.15.129:0/818670315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe77c082950 con 0x7fe77c109b30 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b970 tx=0x7fe77400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe76b7fe640 1 -- 172.21.15.129:0/818670315 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe774019070 con 0x7fe77c109b30 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe76b7fe640 1 -- 172.21.15.129:0/818670315 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe774014440 con 0x7fe77c109b30 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe76b7fe640 1 -- 172.21.15.129:0/818670315 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740075d0 con 0x7fe77c109b30 2024-09-05T01:19:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe77c082c40 con 0x7fe77c109b30 2024-09-05T01:19:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.628+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77c071c40 con 0x7fe77c109b30 2024-09-05T01:19:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.632+0000 7fe783a80740 1 Processor -- start 2024-09-05T01:19:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 -- start start 2024-09-05T01:19:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b8713a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2b866f2f0 con 0x55b2b870bb70 2024-09-05T01:19:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8617270 con 0x55b2b8711540 2024-09-05T01:19:00.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.633+0000 7fe783a80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2b866b2b0 con 0x55b2b870c4b0 2024-09-05T01:19:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe76b7fe640 1 -- 172.21.15.129:0/818670315 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe77401c070 con 0x7fe77c109b30 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36526/0 (socket says 172.21.15.129:36526) 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 -- 172.21.15.129:0/1665842265 learned_addr learned my addr 172.21.15.129:0/1665842265 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b8713a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 -- 172.21.15.129:0/1665842265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 msgr2=0x55b2b8713a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b8713a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 -- 172.21.15.129:0/1665842265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 msgr2=0x55b2b870bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.635+0000 7fe77a59c640 1 -- 172.21.15.129:0/1665842265 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b8616340 con 0x55b2b870c4b0 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.636+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe770009b60 tx=0x7fe770010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9f00f78f7521f7b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.636+0000 7fe768ff9640 1 -- 172.21.15.129:0/1665842265 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770010e90 con 0x55b2b870c4b0 2024-09-05T01:19:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.636+0000 7fe768ff9640 1 -- 172.21.15.129:0/1665842265 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe770009e50 con 0x55b2b870c4b0 2024-09-05T01:19:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.636+0000 7fe768ff9640 1 -- 172.21.15.129:0/1665842265 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7700129a0 con 0x55b2b870c4b0 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe76b7fe640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 0x7fe7640a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 -- 172.21.15.129:0/1665842265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 msgr2=0x55b2b8710f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe770009b60 tx=0x7fe770010740 comp rx=0 tx=0).stop 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 -- 172.21.15.129:0/1665842265 shutdown_connections 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b8713a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b8710f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 --2- 172.21.15.129:0/1665842265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe783a80740 1 -- 172.21.15.129:0/1665842265 >> 172.21.15.129:0/1665842265 conn(0x55b2b87071e0 msgr2=0x55b2b87075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe76b7fe640 1 -- 172.21.15.129:0/818670315 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe7740ce960 con 0x7fe77c109b30 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.637+0000 7fe779d9b640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 0x7fe7640a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.638+0000 7fe783a80740 1 -- 172.21.15.129:0/1665842265 shutdown_connections 2024-09-05T01:19:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.638+0000 7fe783a80740 1 -- 172.21.15.129:0/1665842265 wait complete. 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.638+0000 7fe783a80740 1 -- start start 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.638+0000 7fe783a80740 1 Processor -- start 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b8715e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8716360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8714510 con 0x55b2b870bb70 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2b85f8290 con 0x55b2b870c4b0 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2b8714690 con 0x55b2b8711540 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.639+0000 7fe783a80740 10 client.0 _ll_register_callbacks cb 0x55b2b8629140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:00.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b8715e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36538/0 (socket says 172.21.15.129:36538) 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 -- 172.21.15.129:0/1067239575 learned_addr learned my addr 172.21.15.129:0/1067239575 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 -- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 msgr2=0x55b2b8716360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8716360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 -- 172.21.15.129:0/1067239575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 msgr2=0x55b2b8715e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b8715e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.640+0000 7fe77ad9d640 1 -- 172.21.15.129:0/1067239575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b8382210 con 0x55b2b8711540 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe774097720 tx=0x7fe77409b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64f1c5f0662eb729 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7740a3730 con 0x55b2b8711540 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe779d9b640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 0x7fe7640a5180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c030b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe770009e50 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.641+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe77400cb40 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe77400cd80 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 10 client.25252 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2b844e620 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe770009e50 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe774099c10 con 0x55b2b8711540 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 10 client.25252 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 10 client.25252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 20 client.25252 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b8721f00 0x55b2b8724360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 4 client.25252 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.642+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2b867ca20 con 0x55b2b8721f00 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.643+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b8721f00 0x55b2b8724360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.643+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b8721f00 0x55b2b8724360 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e940932f7821f89 server_cookie=e8ffaaa12e3b76ac in_seq=0 out_seq=0 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.644+0000 7fe761ffb640 10 client.25252 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.644+0000 7fe761ffb640 1 -- 172.21.15.129:0/1067239575 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2b867ca20 con 0x55b2b8721f00 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.644+0000 7fe761ffb640 10 client.25252 handle_command_reply: tid=0 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.644+0000 7fe783a80740 1 client.25252 shutdown 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 msgr2=0x55b2b871d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe774097720 tx=0x7fe77409b590 comp rx=0 tx=0).stop 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 shutdown_connections 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b8721f00 0x55b2b8724360 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871d3a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8716360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 --2- 172.21.15.129:0/1067239575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b8715e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.645+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 >> 172.21.15.129:0/1067239575 conn(0x55b2b87071e0 msgr2=0x55b2b87089f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.646+0000 7fe761ffb640 0 client.25252 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.646+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 shutdown_connections 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.646+0000 7fe783a80740 1 -- 172.21.15.129:0/1067239575 wait complete. 2024-09-05T01:19:00.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.646+0000 7fe783a80740 20 client.25252 trim_cache size 0 max 16384 2024-09-05T01:19:00.667 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:19:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.673+0000 7fe783a80740 1 Processor -- start 2024-09-05T01:19:00.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.673+0000 7fe783a80740 1 -- start start 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe783a80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe783a80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b871d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53596/0 (socket says 172.21.15.129:53596) 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe77a59c640 1 -- 172.21.15.129:0/1789034608 learned_addr learned my addr 172.21.15.129:0/1789034608 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe783a80740 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b870dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.674+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b871d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c012760 con 0x55b2b870bb70 2024-09-05T01:19:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b870dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c010030 con 0x55b2b8711540 2024-09-05T01:19:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c00de80 con 0x55b2b870c4b0 2024-09-05T01:19:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe779d9b640 1 -- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 msgr2=0x55b2b870dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b870dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe779d9b640 1 -- 172.21.15.129:0/1789034608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 msgr2=0x55b2b870d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.675+0000 7fe779d9b640 1 -- 172.21.15.129:0/1789034608 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b8707000 con 0x55b2b870c4b0 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe77a59c640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b870dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe779d9b640 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b871d360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009830 tx=0x7fe76c03b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a54e112fbb5efcfb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe763fff640 1 -- 172.21.15.129:0/1789034608 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c037040 con 0x55b2b870c4b0 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe763fff640 1 -- 172.21.15.129:0/1789034608 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c051070 con 0x55b2b870c4b0 2024-09-05T01:19:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.676+0000 7fe763fff640 1 -- 172.21.15.129:0/1789034608 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe76c032710 con 0x55b2b870c4b0 2024-09-05T01:19:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 msgr2=0x55b2b871d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b871d360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009830 tx=0x7fe76c03b000 comp rx=0 tx=0).stop 2024-09-05T01:19:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 shutdown_connections 2024-09-05T01:19:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b8711540 0x55b2b870dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b870c4b0 0x55b2b871d360 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 --2- 172.21.15.129:0/1789034608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b870d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 >> 172.21.15.129:0/1789034608 conn(0x55b2b87071e0 msgr2=0x55b2b8710d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 shutdown_connections 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.677+0000 7fe783a80740 1 -- 172.21.15.129:0/1789034608 wait complete. 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 -- start start 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 Processor -- start 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b871d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8719f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c00e4f0 con 0x55b2b870bb70 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c00f960 con 0x55b2b870c4b0 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe75c00ebc0 con 0x55b2b8711540 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.678+0000 7fe783a80740 10 client.0 _ll_register_callbacks cb 0x7fe774097070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.679+0000 7fe779d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8719f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.679+0000 7fe779d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8719f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59000/0 (socket says 172.21.15.129:59000) 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.679+0000 7fe779d9b640 1 -- 172.21.15.129:0/3951051555 learned_addr learned my addr 172.21.15.129:0/3951051555 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.679+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.679+0000 7fe77a59c640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b871d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.680+0000 7fe77ad9d640 1 -- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 msgr2=0x55b2b8719f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.680+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8719f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.680+0000 7fe77ad9d640 1 -- 172.21.15.129:0/3951051555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 msgr2=0x55b2b871d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.680+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b871d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.680+0000 7fe77ad9d640 1 -- 172.21.15.129:0/3951051555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b878ad00 con 0x55b2b8711540 2024-09-05T01:19:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.681+0000 7fe77ad9d640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871a570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe774014e10 tx=0x7fe77409a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cf77f51a5d6df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.681+0000 7fe761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:19:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.681+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77409ab20 con 0x55b2b8711540 2024-09-05T01:19:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.681+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe76c051070 con 0x55b2b8711540 2024-09-05T01:19:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.682+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe774096aa0 con 0x55b2b8711540 2024-09-05T01:19:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.682+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe774096ca0 con 0x55b2b8711540 2024-09-05T01:19:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.682+0000 7fe783a80740 10 client.25258 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.682+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b2b878c0e0 con 0x55b2b8711540 2024-09-05T01:19:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe76c051070 con 0x55b2b8711540 2024-09-05T01:19:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe774014870 con 0x55b2b8711540 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 10 client.25258 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 20 client.25258 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b878ca90 0x55b2b878eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 4 client.25258 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2b8676d70 con 0x55b2b878ca90 2024-09-05T01:19:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.683+0000 7fe77a59c640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b878ca90 0x55b2b878eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.685+0000 7fe77a59c640 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b878ca90 0x55b2b878eeb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0542a0349eda52 server_cookie=e4731d41f142f78c in_seq=0 out_seq=0 2024-09-05T01:19:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.685+0000 7fe761ffb640 10 client.25258 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.719+0000 7fe761ffb640 1 -- 172.21.15.129:0/3951051555 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe77002d040 con 0x55b2b878ca90 2024-09-05T01:19:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe761ffb640 10 client.25258 handle_command_reply: tid=0 2024-09-05T01:19:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 client.25258 shutdown 2024-09-05T01:19:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 msgr2=0x55b2b871a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871a570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe774014e10 tx=0x7fe77409a890 comp rx=0 tx=0).stop 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 shutdown_connections 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2b878ca90 0x55b2b878eeb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2b8711540 0x55b2b871a570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2b870c4b0 0x55b2b8719f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 --2- 172.21.15.129:0/3951051555 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2b870bb70 0x55b2b871d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.720+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 >> 172.21.15.129:0/3951051555 conn(0x55b2b87071e0 msgr2=0x55b2b87108d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.721+0000 7fe761ffb640 0 client.25258 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.721+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 shutdown_connections 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.721+0000 7fe783a80740 1 -- 172.21.15.129:0/3951051555 wait complete. 2024-09-05T01:19:00.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.721+0000 7fe783a80740 20 client.25258 trim_cache size 0 max 16384 2024-09-05T01:19:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:19:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:19:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:19:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:19:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.722+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 msgr2=0x7fe7640a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.722+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 0x7fe7640a5180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000c00 tx=0x7fe76c030b50 comp rx=0 tx=0).stop 2024-09-05T01:19:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.722+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 msgr2=0x7fe77c082220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.722+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe77400b970 tx=0x7fe77400be30 comp rx=0 tx=0).stop 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 shutdown_connections 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe7640a2cc0 0x7fe7640a5180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe77c109b30 0x7fe77c082220 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe77c109170 0x7fe77c081310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 --2- 172.21.15.129:0/818670315 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe77c108410 0x7fe77c080dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.723+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 >> 172.21.15.129:0/818670315 conn(0x7fe77c0771d0 msgr2=0x7fe77c10c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.724+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 shutdown_connections 2024-09-05T01:19:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:00.724+0000 7fe77b59e640 1 -- 172.21.15.129:0/818670315 wait complete. 2024-09-05T01:19:01.150 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:00 smithi129 ceph-mon[30979]: pgmap v462: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 272 KiB/s wr, 44 op/s 2024-09-05T01:19:01.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:00 smithi183 ceph-mon[34589]: pgmap v462: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 272 KiB/s wr, 44 op/s 2024-09-05T01:19:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:00 smithi194 ceph-mon[34941]: pgmap v462: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 272 KiB/s wr, 44 op/s 2024-09-05T01:19:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.288+0000 7f103862d640 1 -- 172.21.15.129:0/384290222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1030109830 msgr2=0x7f1030109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:01.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.288+0000 7f103862d640 1 --2- 172.21.15.129:0/384290222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1030109830 0x7f1030109c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1020009900 tx=0x7f102002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.290+0000 7f103862d640 1 -- 172.21.15.129:0/384290222 shutdown_connections 2024-09-05T01:19:01.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.290+0000 7f103862d640 1 --2- 172.21.15.129:0/384290222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f103011b140 0x7f103011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.290+0000 7f103862d640 1 --2- 172.21.15.129:0/384290222 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1030109830 0x7f1030109c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.290+0000 7f103862d640 1 --2- 172.21.15.129:0/384290222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 0x7f1030110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.290+0000 7f103862d640 1 -- 172.21.15.129:0/384290222 >> 172.21.15.129:0/384290222 conn(0x7f103006e140 msgr2=0x7f103006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:01.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.291+0000 7f103862d640 1 -- 172.21.15.129:0/384290222 shutdown_connections 2024-09-05T01:19:01.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.291+0000 7f103862d640 1 -- 172.21.15.129:0/384290222 wait complete. 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.291+0000 7f103862d640 1 Processor -- start 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.291+0000 7f103862d640 1 -- start start 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 0x7f10301a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 0x7f10301a7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120ca0 con 0x7f1030109830 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120b20 con 0x7f1030109450 2024-09-05T01:19:01.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f103862d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1030120e20 con 0x7f103011b140 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f1036bca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f1036bca640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36582/0 (socket says 172.21.15.129:36582) 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f1036bca640 1 -- 172.21.15.129:0/3220152165 learned_addr learned my addr 172.21.15.129:0/3220152165 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.292+0000 7f1035bc8640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 0x7f10301a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 -- 172.21.15.129:0/3220152165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 msgr2=0x7f10301a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f10363c9640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 0x7f10301a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:01.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 0x7f10301a7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 -- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 msgr2=0x7f10301a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:01.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 0x7f10301a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 -- 172.21.15.129:0/3220152165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10301ac1f0 con 0x7f103011b140 2024-09-05T01:19:01.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1035bc8640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 0x7f10301a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:19:01.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.293+0000 7f1036bca640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f102800e970 tx=0x7f102800ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.295+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102800ccc0 con 0x7f103011b140 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.295+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f102800ce60 con 0x7f103011b140 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.295+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028010490 con 0x7f103011b140 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.296+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10301ac4e0 con 0x7f103011b140 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.296+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1030110920 con 0x7f103011b140 2024-09-05T01:19:01.298 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.297+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1030073b00 con 0x7f103011b140 2024-09-05T01:19:01.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.300+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f10280046c0 con 0x7f103011b140 2024-09-05T01:19:01.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.302+0000 7f10277fe640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 0x7f101c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:01.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.302+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f10280cfc70 con 0x7f103011b140 2024-09-05T01:19:01.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.303+0000 7f10363c9640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 0x7f101c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:01.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.304+0000 7f10363c9640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 0x7f101c0a5210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f102c005fd0 tx=0x7f102c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:01.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.305+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f102809c870 con 0x7f103011b140 2024-09-05T01:19:01.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.719+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f103010b470 con 0x7f103011b140 2024-09-05T01:19:01.721 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:19:01.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.721+0000 7f10277fe640 1 -- 172.21.15.129:0/3220152165 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f102809c170 con 0x7f103011b140 2024-09-05T01:19:01.722 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:19:01.722 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:19:01.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 msgr2=0x7f101c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 0x7f101c0a5210 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f102c005fd0 tx=0x7f102c009500 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 msgr2=0x7f10301abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f102800e970 tx=0x7f102800ee30 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 shutdown_connections 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f101c0a2d50 0x7f101c0a5210 secure :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f102c005fd0 tx=0x7f102c009500 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f103011b140 0x7f10301abaf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1030109830 0x7f10301a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 --2- 172.21.15.129:0/3220152165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1030109450 0x7f10301a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.726+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 >> 172.21.15.129:0/3220152165 conn(0x7f103006e140 msgr2=0x7f103011c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.727+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 shutdown_connections 2024-09-05T01:19:01.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:01.728+0000 7f103862d640 1 -- 172.21.15.129:0/3220152165 wait complete. 2024-09-05T01:19:01.982 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:01 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3220152165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:02.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:01 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3220152165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:02.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:01 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3220152165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:02.586 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0' 2024-09-05T01:19:02.587 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 141.75 --await-for 45.0 2024-09-05T01:19:03.026 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:19:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:02 smithi129 ceph-mon[30979]: pgmap v463: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 309 KiB/s wr, 46 op/s 2024-09-05T01:19:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:02 smithi194 ceph-mon[34941]: pgmap v463: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 309 KiB/s wr, 46 op/s 2024-09-05T01:19:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:02 smithi183 ceph-mon[34589]: pgmap v463: 129 pgs: 129 active+clean; 966 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 309 KiB/s wr, 46 op/s 2024-09-05T01:19:03.699 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 -- 172.21.15.129:0/2439342305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77880952d0 msgr2=0x7f77880a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 --2- 172.21.15.129:0/2439342305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77880952d0 0x7f77880a1320 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7790067960 tx=0x7f7790096e10 comp rx=0 tx=0).stop 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 -- 172.21.15.129:0/2439342305 shutdown_connections 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 --2- 172.21.15.129:0/2439342305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77880952d0 0x7f77880a1320 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 --2- 172.21.15.129:0/2439342305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7788094930 0x7f7788094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 --2- 172.21.15.129:0/2439342305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f778809a550 0x7f778809a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.697+0000 7f778f59e640 1 -- 172.21.15.129:0/2439342305 >> 172.21.15.129:0/2439342305 conn(0x7f778800a1f0 msgr2=0x7f778800a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.698+0000 7f778f59e640 1 -- 172.21.15.129:0/2439342305 shutdown_connections 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.698+0000 7f778f59e640 1 -- 172.21.15.129:0/2439342305 wait complete. 2024-09-05T01:19:03.700 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.699+0000 7f778f59e640 1 Processor -- start 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.699+0000 7f778f59e640 1 -- start start 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77880952d0 0x7f778812e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f778809a550 0x7f7788132760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77880a47a0 con 0x7f778809a550 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77880a4620 con 0x7f7788094930 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778f59e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77880a4920 con 0x7f77880952d0 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43322/0 (socket says 172.21.15.129:43322) 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.700+0000 7f778e59c640 1 -- 172.21.15.129:0/2151438175 learned_addr learned my addr 172.21.15.129:0/2151438175 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778ed9d640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f778809a550 0x7f7788132760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.702 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 -- 172.21.15.129:0/2151438175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77880952d0 msgr2=0x7f778812e790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77880952d0 0x7f778812e790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 -- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f778809a550 msgr2=0x7f7788132760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f778809a550 0x7f7788132760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 -- 172.21.15.129:0/2151438175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7788132e60 con 0x7f7788094930 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.701+0000 7f778e59c640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f778400b450 tx=0x7f778400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.702+0000 7f777f7fe640 1 -- 172.21.15.129:0/2151438175 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784015020 con 0x7f7788094930 2024-09-05T01:19:03.703 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.702+0000 7f777f7fe640 1 -- 172.21.15.129:0/2151438175 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7784011d40 con 0x7f7788094930 2024-09-05T01:19:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.702+0000 7f777f7fe640 1 -- 172.21.15.129:0/2151438175 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400f340 con 0x7f7788094930 2024-09-05T01:19:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.702+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7788133150 con 0x7f7788094930 2024-09-05T01:19:03.704 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.702+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778814ba20 con 0x7f7788094930 2024-09-05T01:19:03.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.706+0000 7f777f7fe640 1 -- 172.21.15.129:0/2151438175 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7784018020 con 0x7f7788094930 2024-09-05T01:19:03.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.708+0000 7f777f7fe640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 0x7f77780a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.708 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.708+0000 7f7796d41740 1 Processor -- start 2024-09-05T01:19:03.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.708+0000 7f777f7fe640 1 -- 172.21.15.129:0/2151438175 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f77840cef90 con 0x7f7788094930 2024-09-05T01:19:03.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.708+0000 7f7796d41740 1 -- start start 2024-09-05T01:19:03.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.708+0000 7f778dd9b640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 0x7f77780a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6b50 0x555905fd0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 0x555905fd3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555905f22160 con 0x555905fc6230 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555905ec9720 con 0x555905fc6b50 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f7796d41740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555905f1e120 con 0x555905fd1410 2024-09-05T01:19:03.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f778e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f778ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 0x555905fd3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f778e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49052/0 (socket says 172.21.15.129:49052) 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f778e59c640 1 -- 172.21.15.129:0/1101738181 learned_addr learned my addr 172.21.15.129:0/1101738181 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.709+0000 7f778dd9b640 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6b50 0x555905fd0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 -- 172.21.15.129:0/1101738181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6b50 msgr2=0x555905fd0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6b50 0x555905fd0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 -- 172.21.15.129:0/1101738181 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 msgr2=0x555905fc6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 -- 172.21.15.129:0/1101738181 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555905c93ba0 con 0x555905fd1410 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778e59c640 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778dd9b640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 0x7f77780a5110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f77880948a0 tx=0x7f7780006bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.712 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.710+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 0x555905fd3800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7790051220 tx=0x7f7790050ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56052dc083ea19fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.711+0000 7f777cff9640 1 -- 172.21.15.129:0/1101738181 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790097c10 con 0x555905fd1410 2024-09-05T01:19:03.713 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.711+0000 7f777cff9640 1 -- 172.21.15.129:0/1101738181 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b040 con 0x555905fd1410 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.711+0000 7f777cff9640 1 -- 172.21.15.129:0/1101738181 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779009f670 con 0x555905fd1410 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- 172.21.15.129:0/1101738181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 msgr2=0x555905fd3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 0x555905fd3800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7790051220 tx=0x7f7790050ea0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- 172.21.15.129:0/1101738181 shutdown_connections 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fd1410 0x555905fd3800 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6b50 0x555905fd0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 --2- 172.21.15.129:0/1101738181 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6230 0x555905fc6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- 172.21.15.129:0/1101738181 >> 172.21.15.129:0/1101738181 conn(0x555905fc27c0 msgr2=0x555905fc2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- 172.21.15.129:0/1101738181 shutdown_connections 2024-09-05T01:19:03.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- 172.21.15.129:0/1101738181 wait complete. 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.712+0000 7f7796d41740 1 -- start start 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 Processor -- start 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555905fd4080 con 0x555905fc6b50 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555905eaa900 con 0x555905fd1410 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555905fd4200 con 0x555905fc6230 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f7796d41740 10 client.0 _ll_register_callbacks cb 0x555905ea8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f778ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f778ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43346/0 (socket says 172.21.15.129:43346) 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f778ed9d640 1 -- 172.21.15.129:0/3164251686 learned_addr learned my addr 172.21.15.129:0/3164251686 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:03.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.713+0000 7f778e59c640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 -- 172.21.15.129:0/3164251686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 msgr2=0x555905fce220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fce220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 -- 172.21.15.129:0/3164251686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 msgr2=0x555905fce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 -- 172.21.15.129:0/3164251686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555905eeac50 con 0x555905fd1410 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f778ed9d640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77900637b0 tx=0x7f7790068410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c821242ecf1b9eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f7775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:19:03.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790063230 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f77900633d0 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.714+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779009b040 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.715+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f77900ba050 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.715+0000 7f7796d41740 10 client.25307 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.715+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555905fdfcd0 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.715+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b040 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f77900628a0 con 0x555905fd1410 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 10 client.25307 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 10 client.25307 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 20 client.25307 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:03.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555905fe3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 4 client.25307 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555905f2f890 con 0x555905fe1020 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.716+0000 7f778e59c640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555905fe3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.717+0000 7f7775ffb640 10 client.25307 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.717+0000 7f778e59c640 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555905fe3410 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4057b8317c9d99c2 server_cookie=dec85decedfee284 in_seq=0 out_seq=0 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.718+0000 7f7775ffb640 1 -- 172.21.15.129:0/3164251686 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555905f2f890 con 0x555905fe1020 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.718+0000 7f7775ffb640 10 client.25307 handle_command_reply: tid=0 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.718+0000 7f7796d41740 1 client.25307 shutdown 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 msgr2=0x555905fceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77900637b0 tx=0x7f7790068410 comp rx=0 tx=0).stop 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 shutdown_connections 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555905fe3410 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fceca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7775ffb640 0 client.25307 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 --2- 172.21.15.129:0/3164251686 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fce220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.719+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 >> 172.21.15.129:0/3164251686 conn(0x555905fc27c0 msgr2=0x555905fc84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.720+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 shutdown_connections 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.720+0000 7f7796d41740 1 -- 172.21.15.129:0/3164251686 wait complete. 2024-09-05T01:19:03.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.721+0000 7f7796d41740 20 client.25307 trim_cache size 0 max 16384 2024-09-05T01:19:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 Processor -- start 2024-09-05T01:19:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 -- start start 2024-09-05T01:19:03.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fe89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fe8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7788153e90 con 0x555905fc6b50 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7788151760 con 0x555905fd1410 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f7796d41740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f778814f5d0 con 0x555905fc6230 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f778ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f778ed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43350/0 (socket says 172.21.15.129:43350) 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.753+0000 7f778ed9d640 1 -- 172.21.15.129:0/1350159524 learned_addr learned my addr 172.21.15.129:0/1350159524 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778e59c640 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fe89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 -- 172.21.15.129:0/1350159524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 msgr2=0x555905fe89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fe89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 -- 172.21.15.129:0/1350159524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 msgr2=0x555905fe8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.755 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fe8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 -- 172.21.15.129:0/1350159524 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555905fc25f0 con 0x555905fd1410 2024-09-05T01:19:03.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f778ed9d640 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7790051790 tx=0x7f7790097390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21599cf87f76eef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.754+0000 7f7777fff640 1 -- 172.21.15.129:0/1350159524 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790062830 con 0x555905fd1410 2024-09-05T01:19:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.756+0000 7f7777fff640 1 -- 172.21.15.129:0/1350159524 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b040 con 0x555905fd1410 2024-09-05T01:19:03.757 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.757+0000 7f7777fff640 1 -- 172.21.15.129:0/1350159524 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779009e910 con 0x555905fd1410 2024-09-05T01:19:03.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.757+0000 7f7796d41740 1 -- 172.21.15.129:0/1350159524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 msgr2=0x555905fcb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:03.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.757+0000 7f7796d41740 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7790051790 tx=0x7f7790097390 comp rx=0 tx=0).stop 2024-09-05T01:19:03.758 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.758+0000 7f7796d41740 1 -- 172.21.15.129:0/1350159524 shutdown_connections 2024-09-05T01:19:03.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.758+0000 7f7796d41740 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fd1410 0x555905fcb480 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.758+0000 7f7796d41740 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fc6b50 0x555905fe8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.758+0000 7f7796d41740 1 --2- 172.21.15.129:0/1350159524 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6230 0x555905fe89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.759 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.758+0000 7f7796d41740 1 -- 172.21.15.129:0/1350159524 >> 172.21.15.129:0/1350159524 conn(0x555905fc27c0 msgr2=0x555905fc94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:03.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.759+0000 7f7796d41740 1 -- 172.21.15.129:0/1350159524 shutdown_connections 2024-09-05T01:19:03.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.759+0000 7f7796d41740 1 -- 172.21.15.129:0/1350159524 wait complete. 2024-09-05T01:19:03.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.759+0000 7f7796d41740 1 -- start start 2024-09-05T01:19:03.760 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 Processor -- start 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 0x555905fd9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 0x555905fd4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fd1410 0x555905fd4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f778814fc20 con 0x555905fd1410 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7788151090 con 0x555905fc6230 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f778e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 0x555905fd9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77881502f0 con 0x555905fc6b50 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f778e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 0x555905fd9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43358/0 (socket says 172.21.15.129:43358) 2024-09-05T01:19:03.761 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f778e59c640 1 -- 172.21.15.129:0/147978110 learned_addr learned my addr 172.21.15.129:0/147978110 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:03.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f7796d41740 10 client.0 _ll_register_callbacks cb 0x7f7784095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:03.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f778dd9b640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 0x555905fd4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.760+0000 7f778ed9d640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fd1410 0x555905fd4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 -- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 msgr2=0x555905fd9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 0x555905fd9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 -- 172.21.15.129:0/147978110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fd1410 msgr2=0x555905fd4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fd1410 0x555905fd4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 -- 172.21.15.129:0/147978110 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555905fc3450 con 0x555905fc6b50 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f778dd9b640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 0x555905fd4490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555905fcf7d0 tx=0x7f7780012760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6e0f15898ac8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.761+0000 7f7775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.762+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780022020 con 0x555905fc6b50 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.762+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779009b040 con 0x555905fc6b50 2024-09-05T01:19:03.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.762+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f778001e470 con 0x555905fc6b50 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.763+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f778002d050 con 0x555905fc6b50 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.763+0000 7f7796d41740 10 client.25276 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.763+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555905fbf800 con 0x555905fc6b50 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.763+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f779009b040 con 0x555905fc6b50 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7780012070 con 0x555905fc6b50 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 10 client.25276 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 10 client.25276 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:19:03.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 20 client.25276 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555906056990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 4 client.25276 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}] 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.75, "await_for": 45.0}) -- 0x555905f29be0 con 0x555905fe1020 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.764+0000 7f778e59c640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555906056990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.765+0000 7f7775ffb640 10 client.25276 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:03.766 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:03.765+0000 7f778e59c640 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555906056990 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aabf83655bb58eb server_cookie=874a2fe191f49200 in_seq=0 out_seq=0 2024-09-05T01:19:04.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7775ffb640 1 -- 172.21.15.129:0/147978110 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555905f29be0 con 0x555905fe1020 2024-09-05T01:19:04.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7775ffb640 10 client.25276 handle_command_reply: tid=0 2024-09-05T01:19:04.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 client.25276 shutdown 2024-09-05T01:19:04.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 msgr2=0x555905fd4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:04.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 0x555905fd4490 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x555905fcf7d0 tx=0x7f7780012760 comp rx=0 tx=0).stop 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 shutdown_connections 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555905fe1020 0x555906056990 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555905fd1410 0x555905fd4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555905fc6b50 0x555905fd4490 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7775ffb640 0 client.25276 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 --2- 172.21.15.129:0/147978110 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555905fc6230 0x555905fd9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 >> 172.21.15.129:0/147978110 conn(0x555905fc27c0 msgr2=0x555905fce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 shutdown_connections 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 1 -- 172.21.15.129:0/147978110 wait complete. 2024-09-05T01:19:04.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.035+0000 7f7796d41740 20 client.25276 trim_cache size 0 max 16384 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 14, 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "59d12f8d": { 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 14, 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:19:04.038 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 141.8, 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:04.039 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:04.040 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:19:04.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 msgr2=0x7f77780a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:04.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 0x7f77780a5110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f77880948a0 tx=0x7f7780006bd0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 msgr2=0x7f778812e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:04.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f778400b450 tx=0x7f778400b910 comp rx=0 tx=0).stop 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 shutdown_connections 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77780a2c50 0x7f77780a5110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f778809a550 0x7f7788132760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77880952d0 0x7f778812e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 --2- 172.21.15.129:0/2151438175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7788094930 0x7f778812e250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 >> 172.21.15.129:0/2151438175 conn(0x7f778800a1f0 msgr2=0x7f7788092130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 shutdown_connections 2024-09-05T01:19:04.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:04.036+0000 7f778f59e640 1 -- 172.21.15.129:0/2151438175 wait complete. 2024-09-05T01:19:04.872 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '59d12f8d', quiesce took 4.7 seconds. Will release after: 17.05 2024-09-05T01:19:05.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 120.0 s, 673.2 tps, lat 106.531 ms stddev 31.476 2024-09-05T01:19:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:04 smithi129 ceph-mon[30979]: pgmap v464: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T01:19:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:04 smithi194 ceph-mon[34941]: pgmap v464: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T01:19:05.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:04 smithi183 ceph-mon[34589]: pgmap v464: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T01:19:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:06 smithi129 ceph-mon[30979]: pgmap v465: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-05T01:19:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:06 smithi194 ceph-mon[34941]: pgmap v465: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-05T01:19:07.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:06 smithi183 ceph-mon[34589]: pgmap v465: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.7 MiB/s wr, 336 op/s 2024-09-05T01:19:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:08 smithi129 ceph-mon[30979]: pgmap v466: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-09-05T01:19:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:08 smithi194 ceph-mon[34941]: pgmap v466: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-09-05T01:19:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:08 smithi183 ceph-mon[34589]: pgmap v466: 129 pgs: 129 active+clean; 966 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.4 MiB/s wr, 415 op/s 2024-09-05T01:19:10.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 125.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:19:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:10 smithi129 ceph-mon[30979]: pgmap v467: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-05T01:19:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:10 smithi194 ceph-mon[34941]: pgmap v467: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-05T01:19:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:10 smithi183 ceph-mon[34589]: pgmap v467: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 3.5 MiB/s wr, 429 op/s 2024-09-05T01:19:12.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:19:12.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:19:12.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:19:12.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:19:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:19:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:19:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:19:12.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:19:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:19:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:19:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:19:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:19:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:12 smithi129 ceph-mon[30979]: pgmap v468: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 422 op/s 2024-09-05T01:19:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:12 smithi194 ceph-mon[34941]: pgmap v468: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 422 op/s 2024-09-05T01:19:13.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:12 smithi183 ceph-mon[34589]: pgmap v468: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.4 MiB/s wr, 422 op/s 2024-09-05T01:19:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/2323712231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 msgr2=0x7f2c641031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/2323712231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c641031d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009900 tx=0x7f2c4c02a410 comp rx=0 tx=0).stop 2024-09-05T01:19:13.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/2323712231 shutdown_connections 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/2323712231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c64109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/2323712231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c641031d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/2323712231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 0x7f2c64101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/2323712231 >> 172.21.15.129:0/2323712231 conn(0x7f2c640fd2e0 msgr2=0x7f2c640ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/2323712231 shutdown_connections 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.839+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/2323712231 wait complete. 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 Processor -- start 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 -- start start 2024-09-05T01:19:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 0x7f2c64197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c6419f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410d670 con 0x7f2c64102d50 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410d4f0 con 0x7f2c64103710 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c6a2e6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6410d7f0 con 0x7f2c64101b60 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45880/0 (socket says 172.21.15.129:45880) 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c637fe640 1 -- 172.21.15.129:0/3707584380 learned_addr learned my addr 172.21.15.129:0/3707584380 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.840+0000 7f2c68883640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c6419f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c63fff640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 0x7f2c64197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 -- 172.21.15.129:0/3707584380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 msgr2=0x7f2c64197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 0x7f2c64197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 -- 172.21.15.129:0/3707584380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 msgr2=0x7f2c6419f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c6419f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 -- 172.21.15.129:0/3707584380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6419f930 con 0x7f2c64102d50 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c68883640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c6419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c637fe640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005940 tx=0x7f2c4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c617fa640 1 -- 172.21.15.129:0/3707584380 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c03f030 con 0x7f2c64102d50 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c617fa640 1 -- 172.21.15.129:0/3707584380 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c003cb0 con 0x7f2c64102d50 2024-09-05T01:19:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6419fbc0 con 0x7f2c64102d50 2024-09-05T01:19:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c617fa640 1 -- 172.21.15.129:0/3707584380 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c035070 con 0x7f2c64102d50 2024-09-05T01:19:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.841+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c641a00b0 con 0x7f2c64102d50 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.843+0000 7f2c617fa640 1 -- 172.21.15.129:0/3707584380 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2c4c042030 con 0x7f2c64102d50 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 Processor -- start 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 -- start start 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c617fa640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 0x7f2c500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1f4780 0x55e8aa1f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa14e2f0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa0f6270 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c6d076740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa14a2b0 con 0x55e8aa1f4780 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c68883640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 0x7f2c500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45890/0 (socket says 172.21.15.129:45890) 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c63fff640 1 -- 172.21.15.129:0/3972535895 learned_addr learned my addr 172.21.15.129:0/3972535895 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c617fa640 1 -- 172.21.15.129:0/3707584380 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f2c4c0efa80 con 0x7f2c64102d50 2024-09-05T01:19:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.844+0000 7f2c637fe640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c68883640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1f4780 0x55e8aa1f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 -- 172.21.15.129:0/3972535895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1f4780 msgr2=0x55e8aa1f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1f4780 0x55e8aa1f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 -- 172.21.15.129:0/3972535895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 msgr2=0x55e8aa1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 -- 172.21.15.129:0/3972535895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8aa0f5340 con 0x55e8aa1eedb0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c637fe640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c68883640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 0x7f2c500a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0099e0 tx=0x7f2c5c00fef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c63fff640 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400cba0 tx=0x7f2c5400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86788210899ab01 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c5a7fc640 1 -- 172.21.15.129:0/3972535895 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54010630 con 0x55e8aa1eedb0 2024-09-05T01:19:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c5a7fc640 1 -- 172.21.15.129:0/3972535895 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c54010c80 con 0x55e8aa1eedb0 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.845+0000 7f2c5a7fc640 1 -- 172.21.15.129:0/3972535895 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5401ab40 con 0x55e8aa1eedb0 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- 172.21.15.129:0/3972535895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 msgr2=0x55e8aa1ef1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400cba0 tx=0x7f2c5400e720 comp rx=0 tx=0).stop 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- 172.21.15.129:0/3972535895 shutdown_connections 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1f4780 0x55e8aa1f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 --2- 172.21.15.129:0/3972535895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1ef1b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- 172.21.15.129:0/3972535895 >> 172.21.15.129:0/3972535895 conn(0x55e8aa1e6010 msgr2=0x55e8aa1e6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- 172.21.15.129:0/3972535895 shutdown_connections 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- 172.21.15.129:0/3972535895 wait complete. 2024-09-05T01:19:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 -- start start 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.846+0000 7f2c6d076740 1 Processor -- start 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1f8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa1f7750 con 0x55e8aa1f4780 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa0d7290 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e8aa1f78d0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c6d076740 10 client.0 _ll_register_callbacks cb 0x55e8aa108140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45896/0 (socket says 172.21.15.129:45896) 2024-09-05T01:19:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 -- 172.21.15.129:0/1826195186 learned_addr learned my addr 172.21.15.129:0/1826195186 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c637fe640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1f8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 -- 172.21.15.129:0/1826195186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 msgr2=0x55e8aa1f8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1f8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 -- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 msgr2=0x55e8aa1f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 -- 172.21.15.129:0/1826195186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8a9e61210 con 0x55e8aa1f4780 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c68883640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0107e0 tx=0x7f2c5c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c284b088225f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.847+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c026070 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f2c5c004d20 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c54010c80 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c5c031050 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 10 client.15774 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8a9f2d620 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c54010c80 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2c5c01f070 con 0x55e8aa1f4780 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 10 client.15774 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 20 client.15774 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa204c60 0x55e8aa2070b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 4 client.15774 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8aa15ba20 con 0x55e8aa204c60 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.848+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa204c60 0x55e8aa2070b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.849+0000 7f2c437fe640 10 client.15774 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.849+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa204c60 0x55e8aa2070b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81775b7e80547e69 server_cookie=48fd3bc8aecc32ec in_seq=0 out_seq=0 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.849+0000 7f2c437fe640 1 -- 172.21.15.129:0/1826195186 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8aa15ba20 con 0x55e8aa204c60 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.849+0000 7f2c437fe640 10 client.15774 handle_command_reply: tid=0 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.849+0000 7f2c6d076740 1 client.15774 shutdown 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 msgr2=0x55e8aa200530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0107e0 tx=0x7f2c5c004ab0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 shutdown_connections 2024-09-05T01:19:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa204c60 0x55e8aa2070b0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1f4780 0x55e8aa200530 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c437fe640 0 client.15774 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1826195186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1f8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 >> 172.21.15.129:0/1826195186 conn(0x55e8aa1e6010 msgr2=0x55e8aa1e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 shutdown_connections 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 1 -- 172.21.15.129:0/1826195186 wait complete. 2024-09-05T01:19:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.850+0000 7f2c6d076740 20 client.15774 trim_cache size 0 max 16384 2024-09-05T01:19:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.864+0000 7f2c6d076740 1 Processor -- start 2024-09-05T01:19:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.864+0000 7f2c6d076740 1 -- start start 2024-09-05T01:19:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa20b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641aae80 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641a8750 con 0x55e8aa1f4780 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c6d076740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641a65a0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45906/0 (socket says 172.21.15.129:45906) 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 -- 172.21.15.129:0/756978136 learned_addr learned my addr 172.21.15.129:0/756978136 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c63fff640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c68883640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa20b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 -- 172.21.15.129:0/756978136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 msgr2=0x55e8aa1e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1e50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 -- 172.21.15.129:0/756978136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 msgr2=0x55e8aa20b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa20b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 -- 172.21.15.129:0/756978136 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8aa1efbd0 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c68883640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa20b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c637fe640 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0bbbd0 tx=0x7f2c4c0044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2439e60c99758d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c59ffb640 1 -- 172.21.15.129:0/756978136 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c049070 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c59ffb640 1 -- 172.21.15.129:0/756978136 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c0bc5d0 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.865+0000 7f2c59ffb640 1 -- 172.21.15.129:0/756978136 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c03b460 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- 172.21.15.129:0/756978136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 msgr2=0x55e8aa200450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c0bbbd0 tx=0x7f2c4c0044e0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- 172.21.15.129:0/756978136 shutdown_connections 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa20b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa200450 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 --2- 172.21.15.129:0/756978136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- 172.21.15.129:0/756978136 >> 172.21.15.129:0/756978136 conn(0x55e8aa1e6010 msgr2=0x55e8aa1e49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- 172.21.15.129:0/756978136 shutdown_connections 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- 172.21.15.129:0/756978136 wait complete. 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.866+0000 7f2c6d076740 1 -- start start 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 Processor -- start 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1fe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa1ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641a6c10 con 0x55e8aa1eedb0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641a8080 con 0x55e8aa1f4780 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641a72e0 con 0x55e8aa1ef6f0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c6d076740 10 client.0 _ll_register_callbacks cb 0x7f2c5c012070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45910/0 (socket says 172.21.15.129:45910) 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 -- 172.21.15.129:0/1746427656 learned_addr learned my addr 172.21.15.129:0/1746427656 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c637fe640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c68883640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa1ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 -- 172.21.15.129:0/1746427656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1ef6f0 msgr2=0x55e8aa1fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 -- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 msgr2=0x55e8aa1ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa1ff2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 -- 172.21.15.129:0/1746427656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8aa1ff900 con 0x55e8aa1eedb0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c63fff640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c61fae5f7b6e94f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c437fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.867+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c54005450 con 0x55e8aa1eedb0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4c0bc5d0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f2c540055f0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c54035050 con 0x55e8aa1eedb0 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 10 client.15780 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e8aa26fcc0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2c4c0bc5d0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f2c540266c0 con 0x55e8aa1eedb0 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 10 client.15780 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 20 client.15780 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa270410 0x55e8aa272830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 4 client.15780 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:19:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e8aa155d70 con 0x55e8aa270410 2024-09-05T01:19:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.868+0000 7f2c637fe640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa270410 0x55e8aa272830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.869+0000 7f2c437fe640 10 client.15780 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:13.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.869+0000 7f2c637fe640 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa270410 0x55e8aa272830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9604a751bef9b921 server_cookie=27bbe619230ffa48 in_seq=0 out_seq=0 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c437fe640 1 -- 172.21.15.129:0/1746427656 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e8aa155d70 con 0x55e8aa270410 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c437fe640 10 client.15780 handle_command_reply: tid=0 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 client.15780 shutdown 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 msgr2=0x55e8aa1fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54005f40 comp rx=0 tx=0).stop 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 shutdown_connections 2024-09-05T01:19:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e8aa270410 0x55e8aa272830 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e8aa1f4780 0x55e8aa1ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e8aa1ef6f0 0x55e8aa1fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c437fe640 0 client.15780 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 --2- 172.21.15.129:0/1746427656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8aa1eedb0 0x55e8aa1fddf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 >> 172.21.15.129:0/1746427656 conn(0x55e8aa1e6010 msgr2=0x55e8aa1e49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 shutdown_connections 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 1 -- 172.21.15.129:0/1746427656 wait complete. 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.885+0000 7f2c6d076740 20 client.15780 trim_cache size 0 max 16384 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:19:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:19:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:19:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:19:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 msgr2=0x7f2c500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 0x7f2c500a50f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c0099e0 tx=0x7f2c5c00fef0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 msgr2=0x7f2c64198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c005940 tx=0x7f2c4c0042f0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 shutdown_connections 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2c500a2c30 0x7f2c500a50f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c64103710 0x7f2c6419f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c64102d50 0x7f2c64198260 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 --2- 172.21.15.129:0/3707584380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2c64101b60 0x7f2c64197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 >> 172.21.15.129:0/3707584380 conn(0x7f2c640fd2e0 msgr2=0x7f2c64106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 shutdown_connections 2024-09-05T01:19:13.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:13.886+0000 7f2c6a2e6640 1 -- 172.21.15.129:0/3707584380 wait complete. 2024-09-05T01:19:15.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 130.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:19:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:14 smithi129 ceph-mon[30979]: pgmap v469: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-09-05T01:19:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:14 smithi194 ceph-mon[34941]: pgmap v469: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-09-05T01:19:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:14 smithi183 ceph-mon[34589]: pgmap v469: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 3.4 MiB/s wr, 411 op/s 2024-09-05T01:19:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:16 smithi129 ceph-mon[30979]: pgmap v470: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-09-05T01:19:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:16 smithi194 ceph-mon[34941]: pgmap v470: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-09-05T01:19:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:16 smithi183 ceph-mon[34589]: pgmap v470: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-09-05T01:19:19.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:18 smithi129 ceph-mon[30979]: pgmap v471: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1001 KiB/s wr, 124 op/s 2024-09-05T01:19:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:18 smithi194 ceph-mon[34941]: pgmap v471: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1001 KiB/s wr, 124 op/s 2024-09-05T01:19:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:18 smithi183 ceph-mon[34589]: pgmap v471: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 1001 KiB/s wr, 124 op/s 2024-09-05T01:19:20.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 135.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:19:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:20 smithi129 ceph-mon[30979]: pgmap v472: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 KiB/s wr, 48 op/s 2024-09-05T01:19:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:20 smithi194 ceph-mon[34941]: pgmap v472: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 KiB/s wr, 48 op/s 2024-09-05T01:19:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:20 smithi183 ceph-mon[34589]: pgmap v472: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 288 KiB/s wr, 48 op/s 2024-09-05T01:19:21.924 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:19:22.184 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 -- 172.21.15.129:0/884985862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864106a80 msgr2=0x7f9864106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 --2- 172.21.15.129:0/884985862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864106a80 0x7f9864106e60 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f9854009930 tx=0x7f985402a420 comp rx=0 tx=0).stop 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 -- 172.21.15.129:0/884985862 shutdown_connections 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 --2- 172.21.15.129:0/884985862 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864101420 0x7f986410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 --2- 172.21.15.129:0/884985862 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 0x7f9864100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 --2- 172.21.15.129:0/884985862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864106a80 0x7f9864106e60 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 -- 172.21.15.129:0/884985862 >> 172.21.15.129:0/884985862 conn(0x7f98640fc7a0 msgr2=0x7f98640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 -- 172.21.15.129:0/884985862 shutdown_connections 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.669+0000 7f9869b8d640 1 -- 172.21.15.129:0/884985862 wait complete. 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 Processor -- start 2024-09-05T01:19:22.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 -- start start 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 0x7f9864198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864106a80 0x7f986419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9864110d30 con 0x7f9864101420 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9864110bb0 con 0x7f9864106a80 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9869b8d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9864110eb0 con 0x7f9864100a80 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.670+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46472/0 (socket says 172.21.15.129:46472) 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 -- 172.21.15.129:0/1881913008 learned_addr learned my addr 172.21.15.129:0/1881913008 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f98637fe640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 0x7f9864198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:22.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9863fff640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864106a80 0x7f986419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 -- 172.21.15.129:0/1881913008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 msgr2=0x7f9864198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 0x7f9864198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 -- 172.21.15.129:0/1881913008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864106a80 msgr2=0x7f986419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864106a80 0x7f986419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 -- 172.21.15.129:0/1881913008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986419d340 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9862ffd640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f985800c890 tx=0x7f985800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858004220 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98580043c0 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858004a00 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986419d630 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.671+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641a4f10 con 0x7f9864101420 2024-09-05T01:19:22.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.672+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864100ee0 con 0x7f9864101420 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.674+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9858004ba0 con 0x7f9864101420 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.675+0000 7f9860ff9640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 0x7f984c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.675+0000 7f98637fe640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 0x7f984c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.675+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f98580cf3b0 con 0x7f9864101420 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.675+0000 7f98637fe640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 0x7f984c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f9854000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:22.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.678+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9858014030 con 0x7f9864101420 2024-09-05T01:19:22.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.883+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98641996f0 con 0x7f9864101420 2024-09-05T01:19:22.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.884+0000 7f9860ff9640 1 -- 172.21.15.129:0/1881913008 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f98641996f0 con 0x7f9864101420 2024-09-05T01:19:22.885 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:19:22.886 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:19:22.886 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:19:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.886+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 msgr2=0x7f984c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.886+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 0x7f984c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9854009900 tx=0x7f9854000bc0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 msgr2=0x7f98641988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f985800c890 tx=0x7f985800cd50 comp rx=0 tx=0).stop 2024-09-05T01:19:22.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 shutdown_connections 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f984c0a2c00 0x7f984c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9864106a80 0x7f986419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9864101420 0x7f98641988d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 --2- 172.21.15.129:0/1881913008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9864100a80 0x7f9864198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 >> 172.21.15.129:0/1881913008 conn(0x7f98640fc7a0 msgr2=0x7f986406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 shutdown_connections 2024-09-05T01:19:22.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:22.887+0000 7f9869b8d640 1 -- 172.21.15.129:0/1881913008 wait complete. 2024-09-05T01:19:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:22 smithi129 ceph-mon[30979]: pgmap v473: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 197 KiB/s wr, 34 op/s 2024-09-05T01:19:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:22 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1881913008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:22 smithi194 ceph-mon[34941]: pgmap v473: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 197 KiB/s wr, 34 op/s 2024-09-05T01:19:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:22 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1881913008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:22 smithi183 ceph-mon[34589]: pgmap v473: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 197 KiB/s wr, 34 op/s 2024-09-05T01:19:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:22 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1881913008' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:19:23.708 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 59d12f8d --release --await' 2024-09-05T01:19:23.708 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 59d12f8d --release --await 2024-09-05T01:19:23.959 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:19:24.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 -- 172.21.15.129:0/2658173923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 msgr2=0x7f4c5c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/2658173923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c106f60 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009900 tx=0x7f4c4c02a410 comp rx=0 tx=0).stop 2024-09-05T01:19:24.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 -- 172.21.15.129:0/2658173923 shutdown_connections 2024-09-05T01:19:24.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/2658173923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c10ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/2658173923 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/2658173923 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c106f60 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 -- 172.21.15.129:0/2658173923 >> 172.21.15.129:0/2658173923 conn(0x7f4c5c0fba90 msgr2=0x7f4c5c0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 -- 172.21.15.129:0/2658173923 shutdown_connections 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.471+0000 7f4c61a6e640 1 -- 172.21.15.129:0/2658173923 wait complete. 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 Processor -- start 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 -- start start 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c111e50 con 0x7f4c5c106b80 2024-09-05T01:19:24.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c111cd0 con 0x7f4c5c100500 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c61a6e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c111fd0 con 0x7f4c5c0ffb60 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c5bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46492/0 (socket says 172.21.15.129:46492) 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.472+0000 7f4c5bfff640 1 -- 172.21.15.129:0/11206673 learned_addr learned my addr 172.21.15.129:0/11206673 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c53fff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 -- 172.21.15.129:0/11206673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 msgr2=0x7f4c5c198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 -- 172.21.15.129:0/11206673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 msgr2=0x7f4c5c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 -- 172.21.15.129:0/11206673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c19d2e0 con 0x7f4c5c106b80 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:19:24.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c53fff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c5bfff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400e9d0 tx=0x7f4c5400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c597fa640 1 -- 172.21.15.129:0/11206673 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c5400cd80 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c597fa640 1 -- 172.21.15.129:0/11206673 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c54005020 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c19d5d0 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c597fa640 1 -- 172.21.15.129:0/11206673 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c540116d0 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.473+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1a4eb0 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c597fa640 1 -- 172.21.15.129:0/11206673 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4c540051d0 con 0x7f4c5c106b80 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 Processor -- start 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 -- start start 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389de6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565389d4b620 con 0x565389df5650 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565389cf2e50 con 0x565389de6860 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c63e47740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x565389d475e0 con 0x565389de5f40 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c597fa640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 0x7f4c480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46498/0 (socket says 172.21.15.129:46498) 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c5bfff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 0x7f4c480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c53fff640 1 -- 172.21.15.129:0/885000522 learned_addr learned my addr 172.21.15.129:0/885000522 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c53fff640 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389de6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.476+0000 7f4c597fa640 1 -- 172.21.15.129:0/11206673 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4c540cf6c0 con 0x7f4c5c106b80 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 -- 172.21.15.129:0/885000522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 msgr2=0x565389de6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389de6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 -- 172.21.15.129:0/885000522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 msgr2=0x565389df5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 -- 172.21.15.129:0/885000522 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565389ce3d40 con 0x565389df5650 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c5bfff640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 0x7f4c480a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c5409b810 tx=0x7f4c540666e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c53fff640 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4c44014a30 tx=0x7f4c4400c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aee273226e95b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c51ffb640 1 -- 172.21.15.129:0/885000522 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440142d0 con 0x565389df5650 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c51ffb640 1 -- 172.21.15.129:0/885000522 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c440054d0 con 0x565389df5650 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.477+0000 7f4c51ffb640 1 -- 172.21.15.129:0/885000522 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440047a0 con 0x565389df5650 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- 172.21.15.129:0/885000522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 msgr2=0x565389df7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4c44014a30 tx=0x7f4c4400c570 comp rx=0 tx=0).stop 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- 172.21.15.129:0/885000522 shutdown_connections 2024-09-05T01:19:24.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df7a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 --2- 172.21.15.129:0/885000522 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389de6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- 172.21.15.129:0/885000522 >> 172.21.15.129:0/885000522 conn(0x565389de25f0 msgr2=0x565389de29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- 172.21.15.129:0/885000522 shutdown_connections 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- 172.21.15.129:0/885000522 wait complete. 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 -- start start 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.478+0000 7f4c63e47740 1 Processor -- start 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389df2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565389df8220 con 0x565389df5650 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565389cd3f10 con 0x565389de6860 2024-09-05T01:19:24.509 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x565389df83a0 con 0x565389de5f40 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c63e47740 10 client.0 _ll_register_callbacks cb 0x565389cf7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46500/0 (socket says 172.21.15.129:46500) 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 -- 172.21.15.129:0/2747373199 learned_addr learned my addr 172.21.15.129:0/2747373199 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389df2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 -- 172.21.15.129:0/2747373199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 msgr2=0x565389df2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389df2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 -- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 msgr2=0x565389df2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 -- 172.21.15.129:0/2747373199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565389a5a580 con 0x565389df5650 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c44001eb0 tx=0x7f4c4400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba7e98a0b0ee3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.510 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.479+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440142d0 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4c44012450 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c440054d0 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c44031050 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 10 client.15798 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565389b24960 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c440054d0 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4c44011540 con 0x565389df5650 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 10 client.15798 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 10 client.15798 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 20 client.15798 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e05170 0x565389e07560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.511 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 4 client.15798 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565389d58b90 con 0x565389e05170 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.480+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e05170 0x565389e07560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.481+0000 7f4c3affd640 10 client.15798 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.481+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e05170 0x565389e07560 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45184774e59692a5 server_cookie=5c8d1a64f54cb5ba in_seq=0 out_seq=0 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.481+0000 7f4c3affd640 1 -- 172.21.15.129:0/2747373199 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565389d58b90 con 0x565389e05170 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.481+0000 7f4c3affd640 10 client.15798 handle_command_reply: tid=0 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.481+0000 7f4c63e47740 1 client.15798 shutdown 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 msgr2=0x565389df3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f4c44001eb0 tx=0x7f4c4400cce0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 shutdown_connections 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e05170 0x565389e07560 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.512 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389df5650 0x565389df3010 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2747373199 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389de5f40 0x565389df2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 >> 172.21.15.129:0/2747373199 conn(0x565389de25f0 msgr2=0x565389de8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c3affd640 0 client.15798 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 shutdown_connections 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 1 -- 172.21.15.129:0/2747373199 wait complete. 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.482+0000 7f4c63e47740 20 client.15798 trim_cache size 0 max 16384 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.498+0000 7f4c63e47740 1 Processor -- start 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.498+0000 7f4c63e47740 1 -- start start 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389e0d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389def7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.513 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1af490 con 0x565389de5f40 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1acd60 con 0x565389de6860 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c63e47740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1aabb0 con 0x565389df5650 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46506/0 (socket says 172.21.15.129:46506) 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 -- 172.21.15.129:0/2736911779 learned_addr learned my addr 172.21.15.129:0/2736911779 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 -- 172.21.15.129:0/2736911779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 msgr2=0x565389def7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389def7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 -- 172.21.15.129:0/2736911779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 msgr2=0x565389e0d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389e0d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 -- 172.21.15.129:0/2736911779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565389df4ca0 con 0x565389de5f40 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c53fff640 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c44011940 tx=0x7f4c44005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa27c594299ded29 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.514 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c517fa640 1 -- 172.21.15.129:0/2736911779 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44011cb0 con 0x565389de5f40 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c517fa640 1 -- 172.21.15.129:0/2736911779 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44024020 con 0x565389de5f40 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.499+0000 7f4c517fa640 1 -- 172.21.15.129:0/2736911779 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440128f0 con 0x565389de5f40 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- 172.21.15.129:0/2736911779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 msgr2=0x565389e0cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4c44011940 tx=0x7f4c44005b90 comp rx=0 tx=0).stop 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- 172.21.15.129:0/2736911779 shutdown_connections 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389def7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389e0d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 --2- 172.21.15.129:0/2736911779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389e0cc50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- 172.21.15.129:0/2736911779 >> 172.21.15.129:0/2736911779 conn(0x565389de25f0 msgr2=0x565389de1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- 172.21.15.129:0/2736911779 shutdown_connections 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- 172.21.15.129:0/2736911779 wait complete. 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.500+0000 7f4c63e47740 1 -- start start 2024-09-05T01:19:24.515 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 Processor -- start 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389df9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1ab220 con 0x565389de5f40 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1ac690 con 0x565389de6860 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c1ab8f0 con 0x565389df5650 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c63e47740 10 client.0 _ll_register_callbacks cb 0x7f4c44014930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46514/0 (socket says 172.21.15.129:46514) 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 -- 172.21.15.129:0/551037320 learned_addr learned my addr 172.21.15.129:0/551037320 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.516 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 -- 172.21.15.129:0/551037320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 msgr2=0x565389df9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389df9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 -- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 msgr2=0x565389df8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 -- 172.21.15.129:0/551037320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565389df9860 con 0x565389de5f40 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c53fff640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005b40 tx=0x7f4c440149e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fcd03b71a4b2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.501+0000 7f4c3affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c440140b0 con 0x565389de5f40 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4c44014250 con 0x565389de5f40 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c44024020 con 0x565389de5f40 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4c44011440 con 0x565389de5f40 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 10 client.15804 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:24.517 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x565389e6f950 con 0x565389de5f40 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c44024020 con 0x565389de5f40 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4c4403c030 con 0x565389de5f40 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 10 client.15804 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 10 client.15804 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 20 client.15804 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e6fdc0 0x565389e721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 4 client.15804 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "59d12f8d", "release": true, "await": true}] 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.502+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "59d12f8d", "release": true, "await": true}) -- 0x565389d530a0 con 0x565389e6fdc0 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.503+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e6fdc0 0x565389e721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.503+0000 7f4c3affd640 10 client.15804 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:24.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.503+0000 7f4c5b7fe640 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e6fdc0 0x565389e721b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3b0aa5e58426053 server_cookie=c6e837ccd3b12321 in_seq=0 out_seq=0 2024-09-05T01:19:24.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c3affd640 1 -- 172.21.15.129:0/551037320 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565389d530a0 con 0x565389e6fdc0 2024-09-05T01:19:24.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c3affd640 10 client.15804 handle_command_reply: tid=0 2024-09-05T01:19:24.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 client.15804 shutdown 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 msgr2=0x565389dfda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4c44005b40 tx=0x7f4c440149e0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 shutdown_connections 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565389e6fdc0 0x565389e721b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565389df5650 0x565389df9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565389de6860 0x565389df8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c3affd640 0 client.15804 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 --2- 172.21.15.129:0/551037320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565389de5f40 0x565389dfda60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.588+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 >> 172.21.15.129:0/551037320 conn(0x565389de25f0 msgr2=0x565389de0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.589+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 shutdown_connections 2024-09-05T01:19:24.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.589+0000 7f4c63e47740 1 -- 172.21.15.129:0/551037320 wait complete. 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.589+0000 7f4c63e47740 20 client.15804 trim_cache size 0 max 16384 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 16, 2024-09-05T01:19:24.591 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "59d12f8d": { 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 16, 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 141.8, 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:19:24.592 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:19:24.593 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 msgr2=0x7f4c480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 0x7f4c480a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4c5409b810 tx=0x7f4c540666e0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 msgr2=0x7f4c5c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400e9d0 tx=0x7f4c5400ee90 comp rx=0 tx=0).stop 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 shutdown_connections 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4c480a2c00 0x7f4c480a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c5c106b80 0x7f4c5c19cc00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.594 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4c5c100500 0x7f4c5c198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 --2- 172.21.15.129:0/11206673 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4c5c0ffb60 0x7f4c5c198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:24.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 >> 172.21.15.129:0/11206673 conn(0x7f4c5c0fba90 msgr2=0x7f4c5c0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:24.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 shutdown_connections 2024-09-05T01:19:24.595 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:19:24.590+0000 7f4c61a6e640 1 -- 172.21.15.129:0/11206673 wait complete. 2024-09-05T01:19:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 140.0 s, 127.7 tps, lat 3353.761 ms stddev 7560.801 2024-09-05T01:19:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:24 smithi129 ceph-mon[30979]: pgmap v474: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-09-05T01:19:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:24 smithi194 ceph-mon[34941]: pgmap v474: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-09-05T01:19:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:24 smithi183 ceph-mon[34589]: pgmap v474: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 238 KiB/s wr, 41 op/s 2024-09-05T01:19:25.456 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '59d12f8d', total seconds elapsed: 25.2 2024-09-05T01:19:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.043+0000 7f72ad5c4640 1 -- 172.21.15.129:0/11597636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 msgr2=0x7f72a8119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.043+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/11597636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a8119130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7298009860 tx=0x7f729802a360 comp rx=0 tx=0).stop 2024-09-05T01:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.044+0000 7f72ad5c4640 1 -- 172.21.15.129:0/11597636 shutdown_connections 2024-09-05T01:19:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.044+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/11597636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a8119130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.044+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/11597636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.044+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/11597636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 0x7f72a8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.044+0000 7f72ad5c4640 1 -- 172.21.15.129:0/11597636 >> 172.21.15.129:0/11597636 conn(0x7f72a806ece0 msgr2=0x7f72a8071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.047+0000 7f72ad5c4640 1 -- 172.21.15.129:0/11597636 shutdown_connections 2024-09-05T01:19:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.053+0000 7f72ad5c4640 1 -- 172.21.15.129:0/11597636 wait complete. 2024-09-05T01:19:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.054+0000 7f72ad5c4640 1 Processor -- start 2024-09-05T01:19:27.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.054+0000 7f72ad5c4640 1 -- start start 2024-09-05T01:19:27.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.054+0000 7f72ad5c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72ad5c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a81170f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46534/0 (socket says 172.21.15.129:46534) 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72ad5c4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 0x7f72a8118130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72a6ffd640 1 -- 172.21.15.129:0/3668877269 learned_addr learned my addr 172.21.15.129:0/3668877269 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8110df0 con 0x7f72a8072d20 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8110c70 con 0x7f72a80736c0 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.055+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f72a8110f70 con 0x7f72a8109700 2024-09-05T01:19:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.056+0000 7f72a67fc640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a81170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a77fe640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 0x7f72a8118130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a6ffd640 1 -- 172.21.15.129:0/3668877269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 msgr2=0x7f72a8118130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 0x7f72a8118130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a6ffd640 1 -- 172.21.15.129:0/3668877269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 msgr2=0x7f72a81170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a81170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.058+0000 7f72a6ffd640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a81187a0 con 0x7f72a8072d20 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.059+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d950 tx=0x7f729c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.059+0000 7f728ffff640 1 -- 172.21.15.129:0/3668877269 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c017070 con 0x7f72a8072d20 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.059+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a8118a90 con 0x7f72a8072d20 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.060+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a8118fe0 con 0x7f72a8072d20 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.060+0000 7f728ffff640 1 -- 172.21.15.129:0/3668877269 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f729c009c30 con 0x7f72a8072d20 2024-09-05T01:19:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.060+0000 7f728ffff640 1 -- 172.21.15.129:0/3668877269 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c00f620 con 0x7f72a8072d20 2024-09-05T01:19:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.067+0000 7f728ffff640 1 -- 172.21.15.129:0/3668877269 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f729c00f7c0 con 0x7f72a8072d20 2024-09-05T01:19:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.068+0000 7f72b0439740 1 Processor -- start 2024-09-05T01:19:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.068+0000 7f728ffff640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 0x7f72900a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.069+0000 7f728ffff640 1 -- 172.21.15.129:0/3668877269 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f729c0cef00 con 0x7f72a8072d20 2024-09-05T01:19:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 -- start start 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72a77fe640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 0x7f72900a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd8d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556cbfce92f0 con 0x556cbfd859a0 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556cbfc91270 con 0x556cbfd8b370 2024-09-05T01:19:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72b0439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556cbfce52b0 con 0x556cbfd862e0 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72a77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd8d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd85da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46542/0 (socket says 172.21.15.129:46542) 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.070+0000 7f72a6ffd640 1 -- 172.21.15.129:0/98515577 learned_addr learned my addr 172.21.15.129:0/98515577 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 -- 172.21.15.129:0/98515577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 msgr2=0x556cbfd8ada0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd8ada0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 -- 172.21.15.129:0/98515577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 msgr2=0x556cbfd85da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd85da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 -- 172.21.15.129:0/98515577 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbfc90340 con 0x556cbfd8b370 2024-09-05T01:19:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd8d830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f729802a950 tx=0x7f7298030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbedb3c995f19f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.071+0000 7f72a77fe640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 0x7f72900a5320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7298005b60 tx=0x7f7298009540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.074+0000 7f728d7fa640 1 -- 172.21.15.129:0/98515577 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298037070 con 0x556cbfd8b370 2024-09-05T01:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.074+0000 7f728d7fa640 1 -- 172.21.15.129:0/98515577 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7298038040 con 0x556cbfd8b370 2024-09-05T01:19:27.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.075+0000 7f72b0439740 1 -- 172.21.15.129:0/98515577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 msgr2=0x556cbfd8d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.075+0000 7f72b0439740 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd8d830 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f729802a950 tx=0x7f7298030fc0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.076+0000 7f72b0439740 1 -- 172.21.15.129:0/98515577 shutdown_connections 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.076+0000 7f72b0439740 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd8d830 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.076+0000 7f72b0439740 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd8ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.076+0000 7f72b0439740 1 --2- 172.21.15.129:0/98515577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.076+0000 7f72b0439740 1 -- 172.21.15.129:0/98515577 >> 172.21.15.129:0/98515577 conn(0x556cbfd81010 msgr2=0x556cbfd81410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.078+0000 7f72b0439740 1 -- 172.21.15.129:0/98515577 shutdown_connections 2024-09-05T01:19:27.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.078+0000 7f72b0439740 1 -- 172.21.15.129:0/98515577 wait complete. 2024-09-05T01:19:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.078+0000 7f72b0439740 1 -- start start 2024-09-05T01:19:27.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 Processor -- start 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd8fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd97060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556cbfd8e2d0 con 0x556cbfd862e0 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556cbfc72290 con 0x556cbfd8b370 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556cbfd8e450 con 0x556cbfd859a0 2024-09-05T01:19:27.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.079+0000 7f72b0439740 10 client.0 _ll_register_callbacks cb 0x556cbfca3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.080+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.080+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46552/0 (socket says 172.21.15.129:46552) 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.080+0000 7f72a67fc640 1 -- 172.21.15.129:0/3842071513 learned_addr learned my addr 172.21.15.129:0/3842071513 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.080+0000 7f72a77fe640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd97060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.080+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd8fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.081+0000 7f72a67fc640 1 -- 172.21.15.129:0/3842071513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 msgr2=0x556cbfd8fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.081+0000 7f72a67fc640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd8fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.081+0000 7f72a67fc640 1 -- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 msgr2=0x556cbfd97060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.081+0000 7f72a67fc640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd97060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.082 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.081+0000 7f72a67fc640 1 -- 172.21.15.129:0/3842071513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbf9fc210 con 0x556cbfd862e0 2024-09-05T01:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.082+0000 7f72a67fc640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72a000ea00 tx=0x7f72a000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ac38f00eeea1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.082+0000 7f728aff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.082+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a001c060 con 0x556cbfd862e0 2024-09-05T01:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.082+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7298038040 con 0x556cbfd862e0 2024-09-05T01:19:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.082+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f72a0003970 con 0x556cbfd862e0 2024-09-05T01:19:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.084+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72a0023050 con 0x556cbfd862e0 2024-09-05T01:19:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.084+0000 7f72b0439740 10 client.15816 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.084+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cbfac8620 con 0x556cbfd862e0 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7298038040 con 0x556cbfd862e0 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f72a000b040 con 0x556cbfd862e0 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 10 client.15816 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 20 client.15816 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfd9b7b0 0x556cbfd9dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 4 client.15816 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cbfcf6a20 con 0x556cbfd9b7b0 2024-09-05T01:19:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.085+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfd9b7b0 0x556cbfd9dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.086+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfd9b7b0 0x556cbfd9dbe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb8ca6dd900f5847 server_cookie=40e53a980f40011a in_seq=0 out_seq=0 2024-09-05T01:19:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.087+0000 7f728aff5640 10 client.15816 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.087+0000 7f728aff5640 1 -- 172.21.15.129:0/3842071513 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cbfcf6a20 con 0x556cbfd9b7b0 2024-09-05T01:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.088+0000 7f728aff5640 10 client.15816 handle_command_reply: tid=0 2024-09-05T01:19:27.088 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.088+0000 7f72b0439740 1 client.15816 shutdown 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 msgr2=0x556cbfd90020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f72a000ea00 tx=0x7f72a000ee00 comp rx=0 tx=0).stop 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 shutdown_connections 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfd9b7b0 0x556cbfd9dbe0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd97060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd90020 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 --2- 172.21.15.129:0/3842071513 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd8fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 >> 172.21.15.129:0/3842071513 conn(0x556cbfd81010 msgr2=0x556cbfd82820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f728aff5640 0 client.15816 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 shutdown_connections 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 1 -- 172.21.15.129:0/3842071513 wait complete. 2024-09-05T01:19:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.089+0000 7f72b0439740 20 client.15816 trim_cache size 0 max 16384 2024-09-05T01:19:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 Processor -- start 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 -- start start 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd97020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72a6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46566/0 (socket says 172.21.15.129:46566) 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd81e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f72940127f0 con 0x556cbfd859a0 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f72940100c0 con 0x556cbfd8b370 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72b0439740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f729400df10 con 0x556cbfd862e0 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72a6ffd640 1 -- 172.21.15.129:0/1996074551 learned_addr learned my addr 172.21.15.129:0/1996074551 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.123+0000 7f72a67fc640 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd97020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a77fe640 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd81e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 -- 172.21.15.129:0/1996074551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 msgr2=0x556cbfd97020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd97020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 -- 172.21.15.129:0/1996074551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 msgr2=0x556cbfd81e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd81e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 -- 172.21.15.129:0/1996074551 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cbfd8f620 con 0x556cbfd859a0 2024-09-05T01:19:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f729c0cc5e0 tx=0x7f729c0cc460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce51107d1f0d343 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.124+0000 7f728cff9640 1 -- 172.21.15.129:0/1996074551 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0b4070 con 0x556cbfd859a0 2024-09-05T01:19:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.125+0000 7f728cff9640 1 -- 172.21.15.129:0/1996074551 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f729c099440 con 0x556cbfd859a0 2024-09-05T01:19:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.125+0000 7f728cff9640 1 -- 172.21.15.129:0/1996074551 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c0ae030 con 0x556cbfd859a0 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 -- 172.21.15.129:0/1996074551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 msgr2=0x556cbfd9d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f729c0cc5e0 tx=0x7f729c0cc460 comp rx=0 tx=0).stop 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 -- 172.21.15.129:0/1996074551 shutdown_connections 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd81e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd862e0 0x556cbfd97020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 --2- 172.21.15.129:0/1996074551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd859a0 0x556cbfd9d7c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.126+0000 7f72b0439740 1 -- 172.21.15.129:0/1996074551 >> 172.21.15.129:0/1996074551 conn(0x556cbfd81010 msgr2=0x556cbfd873c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.128+0000 7f72b0439740 1 -- 172.21.15.129:0/1996074551 shutdown_connections 2024-09-05T01:19:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.128+0000 7f72b0439740 1 -- 172.21.15.129:0/1996074551 wait complete. 2024-09-05T01:19:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.129+0000 7f72b0439740 1 -- start start 2024-09-05T01:19:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.129+0000 7f72b0439740 1 Processor -- start 2024-09-05T01:19:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.130+0000 7f72b0439740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd92400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.130+0000 7f72b0439740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.130+0000 7f72b0439740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd95910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72a67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46572/0 (socket says 172.21.15.129:46572) 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72a67fc640 1 -- 172.21.15.129:0/1455152245 learned_addr learned my addr 172.21.15.129:0/1455152245 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f729400e580 con 0x556cbfd862e0 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f729400f9f0 con 0x556cbfd8b370 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f729400ec50 con 0x556cbfd859a0 2024-09-05T01:19:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.131+0000 7f72b0439740 10 client.0 _ll_register_callbacks cb 0x7f72a000e660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 -- 172.21.15.129:0/1455152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 msgr2=0x556cbfd92400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd92400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 -- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 msgr2=0x556cbfd95910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd95910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cbfd95f20 con 0x556cbfd862e0 2024-09-05T01:19:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.132+0000 7f72a67fc640 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4b717ee8e589a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.133+0000 7f728aff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.133+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a00177e0 con 0x556cbfd862e0 2024-09-05T01:19:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.133+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f72a0017a80 con 0x556cbfd862e0 2024-09-05T01:19:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.133+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f729c099440 con 0x556cbfd862e0 2024-09-05T01:19:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.134+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72a001c020 con 0x556cbfd862e0 2024-09-05T01:19:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.135+0000 7f72b0439740 10 client.15822 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.135+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556cbfd972c0 con 0x556cbfd862e0 2024-09-05T01:19:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72a002b030 con 0x556cbfd862e0 2024-09-05T01:19:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f72a000b040 con 0x556cbfd862e0 2024-09-05T01:19:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 20 client.15822 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfe06b50 0x556cbfe08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 4 client.15822 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:19:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.136+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f729c099440 con 0x556cbfe06b50 2024-09-05T01:19:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.137+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfe06b50 0x556cbfe08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.138+0000 7f728aff5640 10 client.15822 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.138+0000 7f72a6ffd640 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfe06b50 0x556cbfe08f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ee5e861d473103 server_cookie=dacd2fd2becc3ceb in_seq=0 out_seq=0 2024-09-05T01:19:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.162+0000 7f728aff5640 1 -- 172.21.15.129:0/1455152245 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f729c099440 con 0x556cbfe06b50 2024-09-05T01:19:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.162+0000 7f728aff5640 10 client.15822 handle_command_reply: tid=0 2024-09-05T01:19:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.162+0000 7f72b0439740 1 client.15822 shutdown 2024-09-05T01:19:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.162+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 msgr2=0x556cbfd92940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.162+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f72a0006010 tx=0x7f72a0005df0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 shutdown_connections 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556cbfe06b50 0x556cbfe08f70 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cbfd8b370 0x556cbfd95910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cbfd862e0 0x556cbfd92940 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 --2- 172.21.15.129:0/1455152245 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556cbfd859a0 0x556cbfd92400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 >> 172.21.15.129:0/1455152245 conn(0x556cbfd81010 msgr2=0x556cbfd7f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.163+0000 7f728aff5640 0 client.15822 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.164+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 shutdown_connections 2024-09-05T01:19:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.164+0000 7f72b0439740 1 -- 172.21.15.129:0/1455152245 wait complete. 2024-09-05T01:19:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.164+0000 7f72b0439740 20 client.15822 trim_cache size 0 max 16384 2024-09-05T01:19:27.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.165+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 msgr2=0x7f72900a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.165+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 0x7f72900a5320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7298005b60 tx=0x7f7298009540 comp rx=0 tx=0).stop 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.165+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 msgr2=0x7f72a8116bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:27.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.165+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f729c00d950 tx=0x7f729c00de10 comp rx=0 tx=0).stop 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 shutdown_connections 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f72900a2e60 0x7f72900a5320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f72a8109700 0x7f72a8118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f72a80736c0 0x7f72a81170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 --2- 172.21.15.129:0/3668877269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f72a8072d20 0x7f72a8116bb0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.166+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 >> 172.21.15.129:0/3668877269 conn(0x7f72a806ece0 msgr2=0x7f72a810c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:27.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.167+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 shutdown_connections 2024-09-05T01:19:27.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:27.167+0000 7f72ad5c4640 1 -- 172.21.15.129:0/3668877269 wait complete. 2024-09-05T01:19:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:26 smithi129 ceph-mon[30979]: pgmap v475: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 35 op/s 2024-09-05T01:19:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:26 smithi194 ceph-mon[34941]: pgmap v475: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 35 op/s 2024-09-05T01:19:27.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:26 smithi183 ceph-mon[34589]: pgmap v475: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 183 KiB/s wr, 35 op/s 2024-09-05T01:19:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:28 smithi129 ceph-mon[30979]: pgmap v476: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-05T01:19:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:28 smithi194 ceph-mon[34941]: pgmap v476: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-05T01:19:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:28 smithi183 ceph-mon[34589]: pgmap v476: 129 pgs: 129 active+clean; 966 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-09-05T01:19:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 145.0 s, 949.8 tps, lat 105.986 ms stddev 122.373 2024-09-05T01:19:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:30 smithi129 ceph-mon[30979]: pgmap v477: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-05T01:19:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:30 smithi194 ceph-mon[34941]: pgmap v477: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-05T01:19:31.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:30 smithi183 ceph-mon[34589]: pgmap v477: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.0 MiB/s wr, 309 op/s 2024-09-05T01:19:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:32 smithi129 ceph-mon[30979]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-09-05T01:19:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:32 smithi194 ceph-mon[34941]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-09-05T01:19:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:32 smithi183 ceph-mon[34589]: pgmap v478: 129 pgs: 129 active+clean; 982 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 4.5 MiB/s wr, 363 op/s 2024-09-05T01:19:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 150.0 s, 1078.6 tps, lat 92.752 ms stddev 25.625 2024-09-05T01:19:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:34 smithi129 ceph-mon[30979]: pgmap v479: 129 pgs: 129 active+clean; 983 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 618 op/s 2024-09-05T01:19:35.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:34 smithi194 ceph-mon[34941]: pgmap v479: 129 pgs: 129 active+clean; 983 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 618 op/s 2024-09-05T01:19:35.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:34 smithi183 ceph-mon[34589]: pgmap v479: 129 pgs: 129 active+clean; 983 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.5 MiB/s wr, 618 op/s 2024-09-05T01:19:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:36 smithi129 ceph-mon[30979]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 799 op/s 2024-09-05T01:19:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:36 smithi194 ceph-mon[34941]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 799 op/s 2024-09-05T01:19:37.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:36 smithi183 ceph-mon[34589]: pgmap v480: 129 pgs: 129 active+clean; 986 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.4 MiB/s wr, 799 op/s 2024-09-05T01:19:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:37 smithi129 ceph-mon[30979]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.7 MiB/s wr, 954 op/s 2024-09-05T01:19:38.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:37 smithi194 ceph-mon[34941]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.7 MiB/s wr, 954 op/s 2024-09-05T01:19:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:37 smithi183 ceph-mon[34589]: pgmap v481: 129 pgs: 129 active+clean; 986 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.7 MiB/s wr, 954 op/s 2024-09-05T01:19:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 155.0 s, 1078.8 tps, lat 92.443 ms stddev 24.960 2024-09-05T01:19:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.330+0000 7ff6539e8640 1 -- 172.21.15.129:0/988772215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c103d80 msgr2=0x7ff64c104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.330+0000 7ff6539e8640 1 --2- 172.21.15.129:0/988772215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c103d80 0x7ff64c104180 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff648009900 tx=0x7ff64802a3f0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.332+0000 7ff6539e8640 1 -- 172.21.15.129:0/988772215 shutdown_connections 2024-09-05T01:19:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.332+0000 7ff6539e8640 1 --2- 172.21.15.129:0/988772215 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c105930 0x7ff64c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.332+0000 7ff6539e8640 1 --2- 172.21.15.129:0/988772215 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c104f70 0x7ff64c1053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.332+0000 7ff6539e8640 1 --2- 172.21.15.129:0/988772215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c103d80 0x7ff64c104180 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.332+0000 7ff6539e8640 1 -- 172.21.15.129:0/988772215 >> 172.21.15.129:0/988772215 conn(0x7ff64c0ff4e0 msgr2=0x7ff64c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.333+0000 7ff6539e8640 1 -- 172.21.15.129:0/988772215 shutdown_connections 2024-09-05T01:19:40.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.334+0000 7ff6539e8640 1 -- 172.21.15.129:0/988772215 wait complete. 2024-09-05T01:19:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.335+0000 7ff6539e8640 1 Processor -- start 2024-09-05T01:19:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.335+0000 7ff6539e8640 1 -- start start 2024-09-05T01:19:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.335+0000 7ff6539e8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.335+0000 7ff6539e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c104f70 0x7ff64c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff651784640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff651784640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56880/0 (socket says 172.21.15.129:56880) 2024-09-05T01:19:40.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff651784640 1 -- 172.21.15.129:0/2871113375 learned_addr learned my addr 172.21.15.129:0/2871113375 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c105930 0x7ff64c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10f8e0 con 0x7ff64c104f70 2024-09-05T01:19:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10f760 con 0x7ff64c103d80 2024-09-05T01:19:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c10fa60 con 0x7ff64c105930 2024-09-05T01:19:40.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff651f85640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c105930 0x7ff64c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.336+0000 7ff650f83640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c104f70 0x7ff64c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 -- 172.21.15.129:0/2871113375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c105930 msgr2=0x7ff64c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c105930 0x7ff64c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 -- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c104f70 msgr2=0x7ff64c19a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c104f70 0x7ff64c19a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c1a1940 con 0x7ff64c103d80 2024-09-05T01:19:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff651784640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff648009850 tx=0x7ff648003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff6427fc640 1 -- 172.21.15.129:0/2871113375 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64803f070 con 0x7ff64c103d80 2024-09-05T01:19:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff6427fc640 1 -- 172.21.15.129:0/2871113375 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff648031dd0 con 0x7ff64c103d80 2024-09-05T01:19:40.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.337+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c1a1bd0 con 0x7ff64c103d80 2024-09-05T01:19:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.338+0000 7ff6427fc640 1 -- 172.21.15.129:0/2871113375 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648035030 con 0x7ff64c103d80 2024-09-05T01:19:40.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.338+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c1a2110 con 0x7ff64c103d80 2024-09-05T01:19:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.341+0000 7ff65685e740 1 Processor -- start 2024-09-05T01:19:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.341+0000 7ff65685e740 1 -- start start 2024-09-05T01:19:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 0x5645d9c9dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9ca2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 0x5645d9ca5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5645d9c012f0 con 0x5645d9ca3370 2024-09-05T01:19:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645d9ba9270 con 0x5645d9c9e2e0 2024-09-05T01:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff65685e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5645d9bfd2b0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff651f85640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 0x5645d9c9dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff651f85640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 0x5645d9c9dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44178/0 (socket says 172.21.15.129:44178) 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff651f85640 1 -- 172.21.15.129:0/699764557 learned_addr learned my addr 172.21.15.129:0/699764557 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff650f83640 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 0x5645d9ca5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.342+0000 7ff6427fc640 1 -- 172.21.15.129:0/2871113375 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff648042070 con 0x7ff64c103d80 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 -- 172.21.15.129:0/699764557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 msgr2=0x5645d9c9dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 0x5645d9c9dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 -- 172.21.15.129:0/699764557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 msgr2=0x5645d9ca2da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9ca2da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 -- 172.21.15.129:0/699764557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d9ba8340 con 0x5645d9ca3370 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff650f83640 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 0x5645d9ca5830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009c20 tx=0x7ff63c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab03b1c25316b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff6427fc640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 0x7ff6380a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff6427fc640 1 -- 172.21.15.129:0/2871113375 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff6480efd50 con 0x7ff64c103d80 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.343+0000 7ff6377fe640 1 -- 172.21.15.129:0/699764557 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c009dd0 con 0x5645d9ca3370 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.344+0000 7ff6377fe640 1 -- 172.21.15.129:0/699764557 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff63c005020 con 0x5645d9ca3370 2024-09-05T01:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.344+0000 7ff651f85640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 0x7ff6380a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.345+0000 7ff6377fe640 1 -- 172.21.15.129:0/699764557 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c004b20 con 0x5645d9ca3370 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.345+0000 7ff65685e740 1 -- 172.21.15.129:0/699764557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 msgr2=0x5645d9ca5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.345+0000 7ff65685e740 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 0x5645d9ca5830 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009c20 tx=0x7ff63c011810 comp rx=0 tx=0).stop 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.345+0000 7ff651f85640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 0x7ff6380a51f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff644005f40 tx=0x7ff644009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 -- 172.21.15.129:0/699764557 shutdown_connections 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9ca3370 0x5645d9ca5830 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9ca2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 --2- 172.21.15.129:0/699764557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9d9a0 0x5645d9c9dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 -- 172.21.15.129:0/699764557 >> 172.21.15.129:0/699764557 conn(0x5645d9c99010 msgr2=0x5645d9c99410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.346+0000 7ff65685e740 1 -- 172.21.15.129:0/699764557 shutdown_connections 2024-09-05T01:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.347+0000 7ff65685e740 1 -- 172.21.15.129:0/699764557 wait complete. 2024-09-05T01:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.347+0000 7ff65685e740 1 -- start start 2024-09-05T01:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.347+0000 7ff65685e740 1 Processor -- start 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cb7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5645d9ca6390 con 0x5645d9c9d9a0 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5645d9b8a290 con 0x5645d9ca3370 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5645d9ca6510 con 0x5645d9c9e2e0 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff65685e740 10 client.0 _ll_register_callbacks cb 0x5645d9bbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:40.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff650f83640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff650f83640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44184/0 (socket says 172.21.15.129:44184) 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff650f83640 1 -- 172.21.15.129:0/1615341063 learned_addr learned my addr 172.21.15.129:0/1615341063 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff651f85640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cb7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff650f83640 1 -- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 msgr2=0x5645d9cb7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.349+0000 7ff650f83640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cb7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff650f83640 1 -- 172.21.15.129:0/1615341063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 msgr2=0x5645d9cb04a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff650f83640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb04a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff650f83640 1 -- 172.21.15.129:0/1615341063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d9914210 con 0x5645d9c9e2e0 2024-09-05T01:19:40.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff650f83640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff63c01b040 tx=0x7ff63c0056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b42f489447d1be71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63c029020 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff63c005020 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.350+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff63c004ab0 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff63c034050 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 10 client.25321 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645d99e0620 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff63c005020 con 0x5645d9c9e2e0 2024-09-05T01:19:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff63c0304b0 con 0x5645d9c9e2e0 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 10 client.25321 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 10 client.25321 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 20 client.25321 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9caaf20 0x5645d9cad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 4 client.25321 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.351+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645d9c0ea20 con 0x5645d9caaf20 2024-09-05T01:19:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.352+0000 7ff651784640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9caaf20 0x5645d9cad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.352+0000 7ff634ff9640 10 client.25321 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.352+0000 7ff651784640 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9caaf20 0x5645d9cad380 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740c95edfc5de014 server_cookie=6cce8ac924bab0f4 in_seq=0 out_seq=0 2024-09-05T01:19:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.353+0000 7ff634ff9640 1 -- 172.21.15.129:0/1615341063 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645d9c0ea20 con 0x5645d9caaf20 2024-09-05T01:19:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.353+0000 7ff634ff9640 10 client.25321 handle_command_reply: tid=0 2024-09-05T01:19:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.353+0000 7ff65685e740 1 client.25321 shutdown 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 msgr2=0x5645d9cb09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff63c01b040 tx=0x7ff63c0056b0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 shutdown_connections 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9caaf20 0x5645d9cad380 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cb7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb09e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 --2- 172.21.15.129:0/1615341063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 >> 172.21.15.129:0/1615341063 conn(0x5645d9c99010 msgr2=0x5645d9c9a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.354+0000 7ff634ff9640 0 client.25321 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.355+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 shutdown_connections 2024-09-05T01:19:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.355+0000 7ff65685e740 1 -- 172.21.15.129:0/1615341063 wait complete. 2024-09-05T01:19:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.355+0000 7ff65685e740 20 client.25321 trim_cache size 0 max 16384 2024-09-05T01:19:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.375+0000 7ff65685e740 1 Processor -- start 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.375+0000 7ff65685e740 1 -- start start 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cadc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1ab050 con 0x5645d9c9d9a0 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a8920 con 0x5645d9ca3370 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff65685e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a6790 con 0x5645d9c9e2e0 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37494/0 (socket says 172.21.15.129:37494) 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 -- 172.21.15.129:0/2257700945 learned_addr learned my addr 172.21.15.129:0/2257700945 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff650f83640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651f85640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cadc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 -- 172.21.15.129:0/2257700945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 msgr2=0x5645d9cb79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 -- 172.21.15.129:0/2257700945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 msgr2=0x5645d9cadc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cadc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 -- 172.21.15.129:0/2257700945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645d9ca27d0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651f85640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cadc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:19:40.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff650f83640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff651784640 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff64802a9b0 tx=0x7ff6480ed2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fe8a245a4735d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff636ffd640 1 -- 172.21.15.129:0/2257700945 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648049040 con 0x5645d9c9d9a0 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff636ffd640 1 -- 172.21.15.129:0/2257700945 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff64800b0b0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.376+0000 7ff636ffd640 1 -- 172.21.15.129:0/2257700945 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480ba580 con 0x5645d9c9d9a0 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- 172.21.15.129:0/2257700945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 msgr2=0x5645d9c980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff64802a9b0 tx=0x7ff6480ed2b0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- 172.21.15.129:0/2257700945 shutdown_connections 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9ca3370 0x5645d9cadc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 --2- 172.21.15.129:0/2257700945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9c980b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- 172.21.15.129:0/2257700945 >> 172.21.15.129:0/2257700945 conn(0x5645d9c99010 msgr2=0x5645d9c979a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.383 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- 172.21.15.129:0/2257700945 shutdown_connections 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- 172.21.15.129:0/2257700945 wait complete. 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.377+0000 7ff65685e740 1 -- start start 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 Processor -- start 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9ca3370 0x5645d9cb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a6de0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a8250 con 0x5645d9c9e2e0 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1a74b0 con 0x5645d9ca3370 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff65685e740 10 client.0 _ll_register_callbacks cb 0x7ff644009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:40.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37498/0 (socket says 172.21.15.129:37498) 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 -- 172.21.15.129:0/2981311779 learned_addr learned my addr 172.21.15.129:0/2981311779 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff650f83640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651f85640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9ca3370 0x5645d9cb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 -- 172.21.15.129:0/2981311779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9ca3370 msgr2=0x5645d9cb6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9ca3370 0x5645d9cb6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 -- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 msgr2=0x5645d9cb36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff651784640 1 -- 172.21.15.129:0/2981311779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645d9cb6d10 con 0x5645d9c9d9a0 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.378+0000 7ff650f83640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff651784640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff648005c20 tx=0x7ff6480bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eda3277d2e6c275 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:40.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6480c4db0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff64803b680 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff64800b0b0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff6480bc1f0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645d9d2f0f0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff64800b0b0 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff6480cc030 con 0x5645d9c9d9a0 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:40.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 20 client.15837 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9d2f9c0 0x5645d9d31de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 4 client.15837 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645d9ca30f0 con 0x5645d9d2f9c0 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.379+0000 7ff650f83640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9d2f9c0 0x5645d9d31de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.380+0000 7ff650f83640 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9d2f9c0 0x5645d9d31de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74db98792681be16 server_cookie=1a03ead0864aee1b in_seq=0 out_seq=0 2024-09-05T01:19:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.380+0000 7ff634ff9640 10 client.15837 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff634ff9640 1 -- 172.21.15.129:0/2981311779 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645d9ca30f0 con 0x5645d9d2f9c0 2024-09-05T01:19:40.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff634ff9640 10 client.15837 handle_command_reply: tid=0 2024-09-05T01:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff65685e740 1 client.15837 shutdown 2024-09-05T01:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 msgr2=0x5645d9cb31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff648005c20 tx=0x7ff6480bbbd0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.555+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 shutdown_connections 2024-09-05T01:19:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5645d9d2f9c0 0x5645d9d31de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff634ff9640 0 client.15837 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5645d9ca3370 0x5645d9cb6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5645d9c9e2e0 0x5645d9cb36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 --2- 172.21.15.129:0/2981311779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5645d9c9d9a0 0x5645d9cb31a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 >> 172.21.15.129:0/2981311779 conn(0x5645d9c99010 msgr2=0x5645d9c981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 shutdown_connections 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 1 -- 172.21.15.129:0/2981311779 wait complete. 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.556+0000 7ff65685e740 20 client.15837 trim_cache size 0 max 16384 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:19:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 msgr2=0x7ff6380a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 0x7ff6380a51f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff644005f40 tx=0x7ff644009600 comp rx=0 tx=0).stop 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 msgr2=0x7ff64c199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff648009850 tx=0x7ff648003f80 comp rx=0 tx=0).stop 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 shutdown_connections 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff6380a2d30 0x7ff6380a51f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff64c105930 0x7ff64c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff64c104f70 0x7ff64c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 --2- 172.21.15.129:0/2871113375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff64c103d80 0x7ff64c199d30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.557+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 >> 172.21.15.129:0/2871113375 conn(0x7ff64c0ff4e0 msgr2=0x7ff64c1081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.558+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 shutdown_connections 2024-09-05T01:19:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:40.558+0000 7ff6539e8640 1 -- 172.21.15.129:0/2871113375 wait complete. 2024-09-05T01:19:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:40 smithi129 ceph-mon[30979]: pgmap v482: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:19:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:40 smithi194 ceph-mon[34941]: pgmap v482: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:19:41.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:40 smithi183 ceph-mon[34589]: pgmap v482: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:19:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:42 smithi129 ceph-mon[30979]: pgmap v483: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-09-05T01:19:43.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:42 smithi194 ceph-mon[34941]: pgmap v483: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-09-05T01:19:43.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:42 smithi183 ceph-mon[34589]: pgmap v483: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.7 MiB/s wr, 1.01k op/s 2024-09-05T01:19:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 160.0 s, 988.7 tps, lat 100.978 ms stddev 37.894 2024-09-05T01:19:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:44 smithi129 ceph-mon[30979]: pgmap v484: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-05T01:19:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:44 smithi194 ceph-mon[34941]: pgmap v484: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-05T01:19:45.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:44 smithi183 ceph-mon[34589]: pgmap v484: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 10 MiB/s wr, 1.21k op/s 2024-09-05T01:19:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:46 smithi129 ceph-mon[30979]: pgmap v485: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-09-05T01:19:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:46 smithi194 ceph-mon[34941]: pgmap v485: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-09-05T01:19:47.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:46 smithi183 ceph-mon[34589]: pgmap v485: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.8 MiB/s wr, 1.12k op/s 2024-09-05T01:19:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:48 smithi129 ceph-mon[30979]: pgmap v486: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:19:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:48 smithi194 ceph-mon[34941]: pgmap v486: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:19:49.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:48 smithi183 ceph-mon[34589]: pgmap v486: 129 pgs: 129 active+clean; 986 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.1 MiB/s wr, 1.09k op/s 2024-09-05T01:19:50.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 165.0 s, 996.2 tps, lat 100.244 ms stddev 34.544 2024-09-05T01:19:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:50 smithi129 ceph-mon[30979]: pgmap v487: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:50 smithi194 ceph-mon[34941]: pgmap v487: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:51.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:50 smithi183 ceph-mon[34589]: pgmap v487: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:52 smithi129 ceph-mon[30979]: pgmap v488: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-09-05T01:19:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:52 smithi194 ceph-mon[34941]: pgmap v488: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-09-05T01:19:53.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:52 smithi183 ceph-mon[34589]: pgmap v488: 129 pgs: 129 active+clean; 986 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.0 MiB/s wr, 955 op/s 2024-09-05T01:19:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36d7fff640 1 -- 172.21.15.129:0/3607563733 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80307c0 con 0x7f36e0102600 2024-09-05T01:19:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 -- 172.21.15.129:0/3607563733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0102600 msgr2=0x7f36e0108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 --2- 172.21.15.129:0/3607563733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0102600 0x7f36e0108e90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009ca0 tx=0x7f36d802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 -- 172.21.15.129:0/3607563733 shutdown_connections 2024-09-05T01:19:53.713 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 --2- 172.21.15.129:0/3607563733 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0102600 0x7f36e0108e90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 --2- 172.21.15.129:0/3607563733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 0x7f36e01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 --2- 172.21.15.129:0/3607563733 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0100a50 0x7f36e0100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.708+0000 7f36e77fd640 1 -- 172.21.15.129:0/3607563733 >> 172.21.15.129:0/3607563733 conn(0x7f36e00fc1b0 msgr2=0x7f36e00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.709+0000 7f36e77fd640 1 -- 172.21.15.129:0/3607563733 shutdown_connections 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.709+0000 7f36e77fd640 1 -- 172.21.15.129:0/3607563733 wait complete. 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.709+0000 7f36e77fd640 1 Processor -- start 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 -- start start 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0100a50 0x7f36e0110d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 0x7f36e01112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.714 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010b450 con 0x7f36e0101c40 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010b2d0 con 0x7f36e0100a50 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e77fd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36e010b5d0 con 0x7f36e0102600 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:49614/0 (socket says 172.21.15.129:49614) 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 -- 172.21.15.129:0/550890986 learned_addr learned my addr 172.21.15.129:0/550890986 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e4d98640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 0x7f36e01112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5599640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0100a50 0x7f36e0110d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 -- 172.21.15.129:0/550890986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0100a50 msgr2=0x7f36e0110d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0100a50 0x7f36e0110d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 -- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 msgr2=0x7f36e01112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 0x7f36e01112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.710+0000 7f36e5d9a640 1 -- 172.21.15.129:0/550890986 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e0116450 con 0x7f36e0102600 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36e5d9a640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36d67fc640 1 -- 172.21.15.129:0/550890986 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004240 con 0x7f36e0102600 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36d67fc640 1 -- 172.21.15.129:0/550890986 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d8003e10 con 0x7f36e0102600 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36d67fc640 1 -- 172.21.15.129:0/550890986 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8030d50 con 0x7f36e0102600 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e01ae9c0 con 0x7f36e0102600 2024-09-05T01:19:53.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.711+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e01aee60 con 0x7f36e0102600 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.715+0000 7f36d67fc640 1 -- 172.21.15.129:0/550890986 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f36d80305c0 con 0x7f36e0102600 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.715+0000 7f36ea67b740 1 Processor -- start 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.715+0000 7f36ea67b740 1 -- start start 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a63830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a409bf2f0 con 0x555a40a61370 2024-09-05T01:19:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a40967270 con 0x555a40a5c2e0 2024-09-05T01:19:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36ea67b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a409bb2b0 con 0x555a40a5b9a0 2024-09-05T01:19:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36d67fc640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 0x7f36cc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.716+0000 7f36d67fc640 1 -- 172.21.15.129:0/550890986 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f36d80f07b0 con 0x7f36e0102600 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:39730/0 (socket says 172.21.15.129:39730) 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 -- 172.21.15.129:0/4020885502 learned_addr learned my addr 172.21.15.129:0/4020885502 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e4d98640 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e4d98640 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a63830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 -- 172.21.15.129:0/4020885502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 msgr2=0x555a40a5bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a5bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 -- 172.21.15.129:0/4020885502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 msgr2=0x555a40a63830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a63830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.717+0000 7f36e5599640 1 -- 172.21.15.129:0/4020885502 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a40966340 con 0x555a40a5c2e0 2024-09-05T01:19:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.718+0000 7f36e5d9a640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 0x7f36cc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.718+0000 7f36e5599640 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00bca0 tx=0x7f36dc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c5136dabec059b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.718+0000 7f36cb7fe640 1 -- 172.21.15.129:0/4020885502 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc009cb0 con 0x555a40a5c2e0 2024-09-05T01:19:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.718+0000 7f36cb7fe640 1 -- 172.21.15.129:0/4020885502 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36dc009e50 con 0x555a40a5c2e0 2024-09-05T01:19:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.719+0000 7f36cb7fe640 1 -- 172.21.15.129:0/4020885502 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc018480 con 0x555a40a5c2e0 2024-09-05T01:19:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.719+0000 7f36ea67b740 1 -- 172.21.15.129:0/4020885502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 msgr2=0x555a40a60da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.719+0000 7f36ea67b740 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00bca0 tx=0x7f36dc005f90 comp rx=0 tx=0).stop 2024-09-05T01:19:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36e5d9a640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 0x7f36cc0a51f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f36d80b9000 tx=0x7f36d803e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 -- 172.21.15.129:0/4020885502 shutdown_connections 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a63830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a60da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 --2- 172.21.15.129:0/4020885502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a5bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 -- 172.21.15.129:0/4020885502 >> 172.21.15.129:0/4020885502 conn(0x555a40a57010 msgr2=0x555a40a57410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 -- 172.21.15.129:0/4020885502 shutdown_connections 2024-09-05T01:19:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.720+0000 7f36ea67b740 1 -- 172.21.15.129:0/4020885502 wait complete. 2024-09-05T01:19:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 -- start start 2024-09-05T01:19:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 Processor -- start 2024-09-05T01:19:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a65bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a6d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a40a643e0 con 0x555a40a61370 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a40948290 con 0x555a40a5c2e0 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a40a64560 con 0x555a40a5b9a0 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.721+0000 7f36ea67b740 10 client.0 _ll_register_callbacks cb 0x555a40979140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e4d98640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a65bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:39736/0 (socket says 172.21.15.129:39736) 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 -- 172.21.15.129:0/3993620916 learned_addr learned my addr 172.21.15.129:0/3993620916 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 -- 172.21.15.129:0/3993620916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 msgr2=0x555a40a65bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a65bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 -- 172.21.15.129:0/3993620916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 msgr2=0x555a40a6d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a6d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.722+0000 7f36e5599640 1 -- 172.21.15.129:0/3993620916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a406d2210 con 0x555a40a5c2e0 2024-09-05T01:19:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.723+0000 7f36e5599640 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b3c0 tx=0x7f36dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b639ccc640a638c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.723+0000 7f36c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.723+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc026070 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.723+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f36dc0035f0 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.723+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc009e50 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.724+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36dc02f050 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.724+0000 7f36ea67b740 10 client.25382 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.724+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a4079e620 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36dc009e50 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc018ca0 con 0x555a40a5c2e0 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 10 client.25382 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 10 client.25382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 20 client.25382 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40a715b0 0x555a40a739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 4 client.25382 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a409cca20 con 0x555a40a715b0 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.725+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40a715b0 0x555a40a739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.726+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40a715b0 0x555a40a739a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=623d9f8bee762dd6 server_cookie=86b5d4a87c6407ee in_seq=0 out_seq=0 2024-09-05T01:19:53.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.726+0000 7f36c8ff9640 10 client.25382 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.728+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3993620916 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a409cca20 con 0x555a40a715b0 2024-09-05T01:19:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.728+0000 7f36c8ff9640 10 client.25382 handle_command_reply: tid=0 2024-09-05T01:19:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.728+0000 7f36ea67b740 1 client.25382 shutdown 2024-09-05T01:19:53.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 msgr2=0x555a40a66100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00b3c0 tx=0x7f36dc004270 comp rx=0 tx=0).stop 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 shutdown_connections 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40a715b0 0x555a40a739a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a61370 0x555a40a6d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a66100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3993620916 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5b9a0 0x555a40a65bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 >> 172.21.15.129:0/3993620916 conn(0x555a40a57010 msgr2=0x555a40a58820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36c8ff9640 0 client.25382 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 shutdown_connections 2024-09-05T01:19:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.729+0000 7f36ea67b740 1 -- 172.21.15.129:0/3993620916 wait complete. 2024-09-05T01:19:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.730+0000 7f36ea67b740 20 client.25382 trim_cache size 0 max 16384 2024-09-05T01:19:53.761 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.760+0000 7f36ea67b740 1 Processor -- start 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 -- start start 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a61370 0x555a40a786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01235c0 con 0x555a40a5b9a0 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36e0120e90 con 0x555a40a5c2e0 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36ea67b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36e011ece0 con 0x555a40a61370 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36e4d98640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.762 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.761+0000 7f36e4d98640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33648/0 (socket says 172.21.15.129:33648) 2024-09-05T01:19:53.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 -- 172.21.15.129:0/1486461912 learned_addr learned my addr 172.21.15.129:0/1486461912 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a61370 0x555a40a786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e5599640 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a6d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 -- 172.21.15.129:0/1486461912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a61370 msgr2=0x555a40a786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a61370 0x555a40a786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 -- 172.21.15.129:0/1486461912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 msgr2=0x555a40a6d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a6d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 -- 172.21.15.129:0/1486461912 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a40a73d10 con 0x555a40a5b9a0 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36e4d98640 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fd0 tx=0x7f36d0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7533fb167a84d878 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36caffd640 1 -- 172.21.15.129:0/1486461912 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004250 con 0x555a40a5b9a0 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.762+0000 7f36caffd640 1 -- 172.21.15.129:0/1486461912 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d0020cb0 con 0x555a40a5b9a0 2024-09-05T01:19:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36caffd640 1 -- 172.21.15.129:0/1486461912 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d001d670 con 0x555a40a5b9a0 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 -- 172.21.15.129:0/1486461912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 msgr2=0x555a40a78170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009fd0 tx=0x7f36d0013040 comp rx=0 tx=0).stop 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 -- 172.21.15.129:0/1486461912 shutdown_connections 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a61370 0x555a40a786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a5c2e0 0x555a40a6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 --2- 172.21.15.129:0/1486461912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a78170 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.763+0000 7f36ea67b740 1 -- 172.21.15.129:0/1486461912 >> 172.21.15.129:0/1486461912 conn(0x555a40a57010 msgr2=0x555a40a561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.764+0000 7f36ea67b740 1 -- 172.21.15.129:0/1486461912 shutdown_connections 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.764+0000 7f36ea67b740 1 -- 172.21.15.129:0/1486461912 wait complete. 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.764+0000 7f36ea67b740 1 -- start start 2024-09-05T01:19:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.764+0000 7f36ea67b740 1 Processor -- start 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a6b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 0x555a40a6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36e011f350 con 0x555a40a5b9a0 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36e01207c0 con 0x555a40a61370 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f36e011fa20 con 0x555a40a5c2e0 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36ea67b740 10 client.0 _ll_register_callbacks cb 0x7f36dc00bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:49656/0 (socket says 172.21.15.129:49656) 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 -- 172.21.15.129:0/3464589695 learned_addr learned my addr 172.21.15.129:0/3464589695 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a6b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 -- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 msgr2=0x555a40a6c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 0x555a40a6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 0x555a40a6c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 -- 172.21.15.129:0/3464589695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 msgr2=0x555a40a6b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a6b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.765+0000 7f36e5599640 1 -- 172.21.15.129:0/3464589695 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a40a6cb90 con 0x555a40a5c2e0 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 0x555a40a6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36e5599640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0058c0 tx=0x7f36dc004f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db82be475b34fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:19:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc038020 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f36dc0035f0 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d0020cb0 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f36dc043050 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.766+0000 7f36ea67b740 10 client.25345 fetch_fsmap learned FSMap version 30 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555a40adc960 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d0020cb0 con 0x555a40a5c2e0 2024-09-05T01:19:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc018480 con 0x555a40a5c2e0 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 10 client.25345 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 20 client.25345 populate_metadata read hostname 'smithi129' 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40add330 0x555a40adf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 4 client.25345 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.767+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a409c6d70 con 0x555a40add330 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.768+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40add330 0x555a40adf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:19:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.771+0000 7f36e4d98640 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40add330 0x555a40adf750 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd32cf1430aa28b0 server_cookie=87c283284759b85c in_seq=0 out_seq=0 2024-09-05T01:19:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.771+0000 7f36c8ff9640 10 client.25345 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.806+0000 7f36c8ff9640 1 -- 172.21.15.129:0/3464589695 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a409c6d70 con 0x555a40add330 2024-09-05T01:19:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.806+0000 7f36c8ff9640 10 client.25345 handle_command_reply: tid=0 2024-09-05T01:19:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.806+0000 7f36ea67b740 1 client.25345 shutdown 2024-09-05T01:19:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.807+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 msgr2=0x555a40a6b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.807+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36dc0058c0 tx=0x7f36dc004f60 comp rx=0 tx=0).stop 2024-09-05T01:19:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 shutdown_connections 2024-09-05T01:19:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a40add330 0x555a40adf750 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a40a61370 0x555a40a6c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a40a5c2e0 0x555a40a6b5c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 --2- 172.21.15.129:0/3464589695 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a40a5b9a0 0x555a40a6b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 >> 172.21.15.129:0/3464589695 conn(0x555a40a57010 msgr2=0x555a40a57ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.808+0000 7f36c8ff9640 0 client.25345 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:19:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.810+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 shutdown_connections 2024-09-05T01:19:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.810+0000 7f36ea67b740 1 -- 172.21.15.129:0/3464589695 wait complete. 2024-09-05T01:19:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.810+0000 7f36ea67b740 20 client.25345 trim_cache size 0 max 16384 2024-09-05T01:19:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:19:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:19:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:19:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:19:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 msgr2=0x7f36cc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 0x7f36cc0a51f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f36d80b9000 tx=0x7f36d803e040 comp rx=0 tx=0).stop 2024-09-05T01:19:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 msgr2=0x7f36e01ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:19:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006010 tx=0x7f36d8003c20 comp rx=0 tx=0).stop 2024-09-05T01:19:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 shutdown_connections 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f36cc0a2d30 0x7f36cc0a51f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f36e0102600 0x7f36e01ae480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36e0101c40 0x7f36e01112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 --2- 172.21.15.129:0/550890986 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f36e0100a50 0x7f36e0110d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.811+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 >> 172.21.15.129:0/550890986 conn(0x7f36e00fc1b0 msgr2=0x7f36e0105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.812+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 shutdown_connections 2024-09-05T01:19:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:19:53.812+0000 7f36e77fd640 1 -- 172.21.15.129:0/550890986 wait complete. 2024-09-05T01:19:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 170.0 s, 950.0 tps, lat 105.760 ms stddev 37.098 2024-09-05T01:19:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:54 smithi129 ceph-mon[30979]: pgmap v489: 129 pgs: 129 active+clean; 986 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:54 smithi194 ceph-mon[34941]: pgmap v489: 129 pgs: 129 active+clean; 986 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:55.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:54 smithi183 ceph-mon[34589]: pgmap v489: 129 pgs: 129 active+clean; 986 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 10 MiB/s wr, 1.18k op/s 2024-09-05T01:19:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:56 smithi129 ceph-mon[30979]: pgmap v490: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:19:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:56 smithi194 ceph-mon[34941]: pgmap v490: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:19:57.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:56 smithi183 ceph-mon[34589]: pgmap v490: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.7 MiB/s wr, 1.02k op/s 2024-09-05T01:19:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:19:58 smithi129 ceph-mon[30979]: pgmap v491: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 999 op/s 2024-09-05T01:19:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:19:58 smithi194 ceph-mon[34941]: pgmap v491: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 999 op/s 2024-09-05T01:19:59.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:19:58 smithi183 ceph-mon[34589]: pgmap v491: 129 pgs: 129 active+clean; 986 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.3 MiB/s wr, 999 op/s 2024-09-05T01:20:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 175.0 s, 962.0 tps, lat 103.555 ms stddev 34.173 2024-09-05T01:20:01.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:00 smithi183 ceph-mon[34589]: pgmap v492: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:01.213 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:00 smithi183 ceph-mon[34589]: overall HEALTH_OK 2024-09-05T01:20:01.227 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:00 smithi129 ceph-mon[30979]: pgmap v492: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:01.227 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:00 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:20:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:00 smithi194 ceph-mon[34941]: pgmap v492: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:00 smithi194 ceph-mon[34941]: overall HEALTH_OK 2024-09-05T01:20:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:02 smithi129 ceph-mon[30979]: pgmap v493: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 904 op/s 2024-09-05T01:20:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:02 smithi194 ceph-mon[34941]: pgmap v493: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 904 op/s 2024-09-05T01:20:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:02 smithi183 ceph-mon[34589]: pgmap v493: 129 pgs: 129 active+clean; 987 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.5 MiB/s wr, 904 op/s 2024-09-05T01:20:05.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:04 smithi183 ceph-mon[34589]: pgmap v494: 129 pgs: 129 active+clean; 987 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 180.0 s, 932.8 tps, lat 107.407 ms stddev 104.603 2024-09-05T01:20:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:04 smithi129 ceph-mon[30979]: pgmap v494: 129 pgs: 129 active+clean; 987 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:04 smithi194 ceph-mon[34941]: pgmap v494: 129 pgs: 129 active+clean; 987 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.960+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3867206879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0077df0 msgr2=0x7ff0d0078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.960+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3867206879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0077df0 0x7ff0d0078250 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009860 tx=0x7ff0c002a340 comp rx=0 tx=0).stop 2024-09-05T01:20:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3867206879 shutdown_connections 2024-09-05T01:20:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3867206879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d0079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3867206879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0077df0 0x7ff0d0078250 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3867206879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0076bb0 0x7ff0d0076fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3867206879 >> 172.21.15.129:0/3867206879 conn(0x7ff0d00ff4e0 msgr2=0x7ff0d0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.962+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3867206879 shutdown_connections 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.963+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3867206879 wait complete. 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.963+0000 7ff0d81a6640 1 Processor -- start 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.963+0000 7ff0d81a6640 1 -- start start 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0077df0 0x7ff0d010ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:56274/0 (socket says 172.21.15.129:56274) 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d01ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d5f42640 1 -- 172.21.15.129:0/3247868493 learned_addr learned my addr 172.21.15.129:0/3247868493 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0113cf0 con 0x7ff0d0077df0 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0113b70 con 0x7ff0d0078790 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0113e70 con 0x7ff0d0076bb0 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.964+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0077df0 0x7ff0d010ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d6743640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d01ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 -- 172.21.15.129:0/3247868493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 msgr2=0x7ff0d01ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d01ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 -- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0077df0 msgr2=0x7ff0d010ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0077df0 0x7ff0d010ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01ae900 con 0x7ff0d0076bb0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d6743640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d01ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9f0 tx=0x7ff0cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0c6ffd640 1 -- 172.21.15.129:0/3247868493 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc00cd80 con 0x7ff0d0076bb0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0c6ffd640 1 -- 172.21.15.129:0/3247868493 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0cc0108e0 con 0x7ff0d0076bb0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0c6ffd640 1 -- 172.21.15.129:0/3247868493 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc018920 con 0x7ff0d0076bb0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.965+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01aead0 con 0x7ff0d0076bb0 2024-09-05T01:20:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.966+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01aee80 con 0x7ff0d0076bb0 2024-09-05T01:20:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.970+0000 7ff0c6ffd640 1 -- 172.21.15.129:0/3247868493 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff0cc018ac0 con 0x7ff0d0076bb0 2024-09-05T01:20:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.971+0000 7ff0c6ffd640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 0x7ff0bc0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.971+0000 7ff0c6ffd640 1 -- 172.21.15.129:0/3247868493 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff0cc0d06d0 con 0x7ff0d0076bb0 2024-09-05T01:20:06.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.972+0000 7ff0db022740 1 Processor -- start 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.972+0000 7ff0db022740 1 -- start start 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1529b0 0x55bffb152db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb158380 0x55bffb15a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bffb0a52f0 con 0x55bffb158380 2024-09-05T01:20:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bffb04d270 con 0x55bffb1532f0 2024-09-05T01:20:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.973+0000 7ff0db022740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bffb0a12b0 con 0x55bffb1529b0 2024-09-05T01:20:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.974+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 0x7ff0bc0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35666/0 (socket says 172.21.15.129:35666) 2024-09-05T01:20:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 -- 172.21.15.129:0/3862874509 learned_addr learned my addr 172.21.15.129:0/3862874509 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1529b0 0x55bffb152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 -- 172.21.15.129:0/3862874509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1529b0 msgr2=0x55bffb152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1529b0 0x55bffb152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 -- 172.21.15.129:0/3862874509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb158380 msgr2=0x55bffb15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb158380 0x55bffb15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.975+0000 7ff0d5741640 1 -- 172.21.15.129:0/3862874509 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bffb04c340 con 0x55bffb1532f0 2024-09-05T01:20:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.976+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0034000 tx=0x7ff0c0045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964811ba51eafbc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.976+0000 7ff0d5741640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 0x7ff0bc0a5330 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009780 tx=0x7ff0c00095b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.977+0000 7ff0bbfff640 1 -- 172.21.15.129:0/3862874509 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0052070 con 0x55bffb1532f0 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.977+0000 7ff0bbfff640 1 -- 172.21.15.129:0/3862874509 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c0038040 con 0x55bffb1532f0 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.977+0000 7ff0bbfff640 1 -- 172.21.15.129:0/3862874509 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0049070 con 0x55bffb1532f0 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.980+0000 7ff0db022740 1 -- 172.21.15.129:0/3862874509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 msgr2=0x55bffb157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.980+0000 7ff0db022740 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0034000 tx=0x7ff0c0045000 comp rx=0 tx=0).stop 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 -- 172.21.15.129:0/3862874509 shutdown_connections 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb158380 0x55bffb15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb157db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 --2- 172.21.15.129:0/3862874509 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1529b0 0x55bffb152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 -- 172.21.15.129:0/3862874509 >> 172.21.15.129:0/3862874509 conn(0x55bffb13d010 msgr2=0x55bffb13d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 -- 172.21.15.129:0/3862874509 shutdown_connections 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.981+0000 7ff0db022740 1 -- 172.21.15.129:0/3862874509 wait complete. 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.982+0000 7ff0db022740 1 -- start start 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.982+0000 7ff0db022740 1 Processor -- start 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.983+0000 7ff0db022740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.983+0000 7ff0db022740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.983+0000 7ff0db022740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.983+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.983+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56916/0 (socket says 172.21.15.129:56916) 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5f42640 1 -- 172.21.15.129:0/789590472 learned_addr learned my addr 172.21.15.129:0/789590472 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bffb15b2f0 con 0x55bffb1529b0 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bffb02e290 con 0x55bffb158380 2024-09-05T01:20:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d6743640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55bffb15b470 con 0x55bffb1532f0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0db022740 10 client.0 _ll_register_callbacks cb 0x55bffb05f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 -- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 msgr2=0x55bffb164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 -- 172.21.15.129:0/789590472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 msgr2=0x55bffb15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bffadb8210 con 0x55bffb1532f0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.984+0000 7ff0d5741640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0c002a930 tx=0x7ff0c0005600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d32e1a01a60a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0052040 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff0c00037f0 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c0038040 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0c0037040 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0db022740 10 client.25357 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.985+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bffae84620 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0c005b070 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff0c004d5c0 con 0x55bffb1532f0 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 10 client.25357 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 10 client.25357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 20 client.25357 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb168470 0x55bffb16a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 4 client.25357 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0c0038040 con 0x55bffb168470 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.986+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb168470 0x55bffb16a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.987+0000 7ff0b97fa640 10 client.25357 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.987+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb168470 0x55bffb16a860 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4716e0eb4221c6 server_cookie=7eaf9ba87b6c0266 in_seq=0 out_seq=0 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.988+0000 7ff0b97fa640 1 -- 172.21.15.129:0/789590472 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff0c0038040 con 0x55bffb168470 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.988+0000 7ff0b97fa640 10 client.25357 handle_command_reply: tid=0 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.989+0000 7ff0db022740 1 client.25357 shutdown 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.989+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 msgr2=0x55bffb15d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.989+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0c002a930 tx=0x7ff0c0005600 comp rx=0 tx=0).stop 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 shutdown_connections 2024-09-05T01:20:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb168470 0x55bffb16a860 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 --2- 172.21.15.129:0/789590472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 >> 172.21.15.129:0/789590472 conn(0x55bffb13d010 msgr2=0x55bffb13e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0b97fa640 0 client.25357 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 shutdown_connections 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 1 -- 172.21.15.129:0/789590472 wait complete. 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:06.990+0000 7ff0db022740 20 client.25357 trim_cache size 0 max 16384 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.018+0000 7ff0db022740 1 Processor -- start 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.018+0000 7ff0db022740 1 -- start start 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.018+0000 7ff0db022740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0db022740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0db022740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0db022740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4012610 con 0x55bffb1529b0 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0db022740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400fee0 con 0x55bffb158380 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0db022740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400dd30 con 0x55bffb1532f0 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:56304/0 (socket says 172.21.15.129:56304) 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56922/0 (socket says 172.21.15.129:56922) 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5741640 1 -- 172.21.15.129:0/900254895 learned_addr learned my addr 172.21.15.129:0/900254895 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d6743640 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 -- 172.21.15.129:0/900254895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 msgr2=0x55bffb15d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 -- 172.21.15.129:0/900254895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 msgr2=0x55bffb164150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.019+0000 7ff0d5f42640 1 -- 172.21.15.129:0/900254895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bffb1577d0 con 0x55bffb1529b0 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.020+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc098f20 tx=0x7ff0cc09f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332820f6d278a42c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.020+0000 7ff0bb7fe640 1 -- 172.21.15.129:0/900254895 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0b9070 con 0x55bffb1529b0 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.020+0000 7ff0bb7fe640 1 -- 172.21.15.129:0/900254895 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0cc0a5730 con 0x55bffb1529b0 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.020+0000 7ff0bb7fe640 1 -- 172.21.15.129:0/900254895 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0bb310 con 0x55bffb1529b0 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 -- 172.21.15.129:0/900254895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 msgr2=0x55bffb15cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc098f20 tx=0x7ff0cc09f040 comp rx=0 tx=0).stop 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 -- 172.21.15.129:0/900254895 shutdown_connections 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb158380 0x55bffb164150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb1532f0 0x55bffb15d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 --2- 172.21.15.129:0/900254895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb15cc10 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 -- 172.21.15.129:0/900254895 >> 172.21.15.129:0/900254895 conn(0x55bffb13d010 msgr2=0x55bffb154530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 -- 172.21.15.129:0/900254895 shutdown_connections 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.021+0000 7ff0db022740 1 -- 172.21.15.129:0/900254895 wait complete. 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.022+0000 7ff0db022740 1 -- start start 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.022+0000 7ff0db022740 1 Processor -- start 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb1615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb158380 0x55bffb162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb1615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d5f42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb1615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56924/0 (socket says 172.21.15.129:56924) 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d5741640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35700/0 (socket says 172.21.15.129:35700) 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d5f42640 1 -- 172.21.15.129:0/464080242 learned_addr learned my addr 172.21.15.129:0/464080242 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0d6743640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb158380 0x55bffb162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400e3a0 con 0x55bffb1529b0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400f810 con 0x55bffb1532f0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b400ea70 con 0x55bffb158380 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.023+0000 7ff0db022740 10 client.0 _ll_register_callbacks cb 0x7ff0c0050070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 -- 172.21.15.129:0/464080242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb158380 msgr2=0x55bffb162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb158380 0x55bffb162ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 -- 172.21.15.129:0/464080242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 msgr2=0x55bffb1615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb1615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bffb1630d0 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0d5741640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0030970 tx=0x7ff0c0030ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ad4c500a26bbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0068020 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff0c004d450 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.024+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0cc0a5730 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff0c0049070 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 10 client.25403 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bffb1643e0 con 0x55bffb1532f0 2024-09-05T01:20:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0cc0a5730 con 0x55bffb1532f0 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff0c00035f0 con 0x55bffb1532f0 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 10 client.25403 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 10 client.25403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 20 client.25403 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb1d3560 0x55bffb1d5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 4 client.25403 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.025+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bffb0b2a20 con 0x55bffb1d3560 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.026+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb1d3560 0x55bffb1d5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.026+0000 7ff0b97fa640 10 client.25403 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.026+0000 7ff0d5f42640 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb1d3560 0x55bffb1d5980 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8bedfb913ce84b server_cookie=e66856cffe99469a in_seq=0 out_seq=0 2024-09-05T01:20:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0b97fa640 1 -- 172.21.15.129:0/464080242 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bffb0b2a20 con 0x55bffb1d3560 2024-09-05T01:20:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0b97fa640 10 client.25403 handle_command_reply: tid=0 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0db022740 1 client.25403 shutdown 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 msgr2=0x55bffb161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0030970 tx=0x7ff0c0030ee0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 shutdown_connections 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.073+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55bffb1d3560 0x55bffb1d5980 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0b97fa640 0 client.25403 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55bffb158380 0x55bffb162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bffb1532f0 0x55bffb161b00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 --2- 172.21.15.129:0/464080242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bffb1529b0 0x55bffb1615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 >> 172.21.15.129:0/464080242 conn(0x55bffb13d010 msgr2=0x55bffb15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 shutdown_connections 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 1 -- 172.21.15.129:0/464080242 wait complete. 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.074+0000 7ff0db022740 20 client.25403 trim_cache size 0 max 16384 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:20:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:20:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.075+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 msgr2=0x7ff0bc0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.075+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 0x7ff0bc0a5330 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009780 tx=0x7ff0c00095b0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.076+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 msgr2=0x7ff0d010e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.076+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00e9f0 tx=0x7ff0cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.076+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 shutdown_connections 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.076+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0bc0a2e70 0x7ff0bc0a5330 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff0d0078790 0x7ff0d01ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0d0077df0 0x7ff0d010ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 --2- 172.21.15.129:0/3247868493 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0d0076bb0 0x7ff0d010e7e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 >> 172.21.15.129:0/3247868493 conn(0x7ff0d00ff4e0 msgr2=0x7ff0d0100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 shutdown_connections 2024-09-05T01:20:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:07.077+0000 7ff0d81a6640 1 -- 172.21.15.129:0/3247868493 wait complete. 2024-09-05T01:20:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:06 smithi129 ceph-mon[30979]: pgmap v495: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.1 MiB/s wr, 973 op/s 2024-09-05T01:20:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:06 smithi194 ceph-mon[34941]: pgmap v495: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.1 MiB/s wr, 973 op/s 2024-09-05T01:20:07.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:06 smithi183 ceph-mon[34589]: pgmap v495: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.1 MiB/s wr, 973 op/s 2024-09-05T01:20:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:08 smithi129 ceph-mon[30979]: pgmap v496: 129 pgs: 129 active+clean; 995 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 1.09k op/s 2024-09-05T01:20:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:08 smithi194 ceph-mon[34941]: pgmap v496: 129 pgs: 129 active+clean; 995 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 1.09k op/s 2024-09-05T01:20:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:08 smithi183 ceph-mon[34589]: pgmap v496: 129 pgs: 129 active+clean; 995 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 9.7 MiB/s wr, 1.09k op/s 2024-09-05T01:20:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 185.0 s, 1047.2 tps, lat 95.400 ms stddev 30.018 2024-09-05T01:20:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:10 smithi129 ceph-mon[30979]: pgmap v497: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:10 smithi194 ceph-mon[34941]: pgmap v497: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:11.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:10 smithi183 ceph-mon[34589]: pgmap v497: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:12.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:11 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:20:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:11 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:20:12.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:11 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:20:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:12 smithi129 ceph-mon[30979]: pgmap v498: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 937 op/s 2024-09-05T01:20:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:20:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:20:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:20:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:12 smithi194 ceph-mon[34941]: pgmap v498: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 937 op/s 2024-09-05T01:20:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:20:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:20:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:20:13.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:12 smithi183 ceph-mon[34589]: pgmap v498: 129 pgs: 129 active+clean; 1003 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 937 op/s 2024-09-05T01:20:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:20:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:20:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:20:15.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 190.0 s, 1006.4 tps, lat 99.188 ms stddev 34.885 2024-09-05T01:20:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:14 smithi194 ceph-mon[34941]: pgmap v499: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-09-05T01:20:15.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:14 smithi183 ceph-mon[34589]: pgmap v499: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-09-05T01:20:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:14 smithi129 ceph-mon[30979]: pgmap v499: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 11 MiB/s wr, 1.12k op/s 2024-09-05T01:20:16.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:15 smithi194 ceph-mon[34941]: pgmap v500: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.6 MiB/s wr, 982 op/s 2024-09-05T01:20:16.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:15 smithi183 ceph-mon[34589]: pgmap v500: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.6 MiB/s wr, 982 op/s 2024-09-05T01:20:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:15 smithi129 ceph-mon[30979]: pgmap v500: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.6 MiB/s wr, 982 op/s 2024-09-05T01:20:19.194 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:18 smithi129 ceph-mon[30979]: pgmap v501: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:18 smithi194 ceph-mon[34941]: pgmap v501: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:18 smithi183 ceph-mon[34589]: pgmap v501: 129 pgs: 129 active+clean; 1003 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 1.13k op/s 2024-09-05T01:20:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 195.0 s, 996.6 tps, lat 100.202 ms stddev 34.408 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 -- 172.21.15.129:0/199786925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581054b0 msgr2=0x7f0758105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 --2- 172.21.15.129:0/199786925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581054b0 0x7f0758105960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0748009900 tx=0x7f074802a3f0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 -- 172.21.15.129:0/199786925 shutdown_connections 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 --2- 172.21.15.129:0/199786925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581054b0 0x7f0758105960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 --2- 172.21.15.129:0/199786925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f0758102b00 0x7f0758104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 --2- 172.21.15.129:0/199786925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07581001b0 0x7f07581025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 -- 172.21.15.129:0/199786925 >> 172.21.15.129:0/199786925 conn(0x7f07580fbfa0 msgr2=0x7f07580fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 -- 172.21.15.129:0/199786925 shutdown_connections 2024-09-05T01:20:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.239+0000 7f075db0b640 1 -- 172.21.15.129:0/199786925 wait complete. 2024-09-05T01:20:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 Processor -- start 2024-09-05T01:20:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 -- start start 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581001b0 0x7f075819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 0x7f075819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0758113aa0 con 0x7f0758102b00 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0758113920 con 0x7f07581001b0 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.240+0000 7f075db0b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0758113c20 con 0x7f07581054b0 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57640/0 (socket says 172.21.15.129:57640) 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 -- 172.21.15.129:0/74156990 learned_addr learned my addr 172.21.15.129:0/74156990 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075cb09640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581001b0 0x7f075819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f0757fff640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 0x7f075819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 -- 172.21.15.129:0/74156990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581001b0 msgr2=0x7f075819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581001b0 0x7f075819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 -- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 msgr2=0x7f075819e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 0x7f075819e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 -- 172.21.15.129:0/74156990 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07581a5d90 con 0x7f07581054b0 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f0757fff640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 0x7f075819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.241+0000 7f075d30a640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0748005ab0 tx=0x7f0748003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.242+0000 7f0755ffb640 1 -- 172.21.15.129:0/74156990 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074803f070 con 0x7f07581054b0 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.242+0000 7f0755ffb640 1 -- 172.21.15.129:0/74156990 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0748031c30 con 0x7f07581054b0 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.242+0000 7f0755ffb640 1 -- 172.21.15.129:0/74156990 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748035030 con 0x7f07581054b0 2024-09-05T01:20:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.242+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07581a6020 con 0x7f07581054b0 2024-09-05T01:20:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.242+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07581a6400 con 0x7f07581054b0 2024-09-05T01:20:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.246+0000 7f0761be3740 1 Processor -- start 2024-09-05T01:20:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.247+0000 7f0761be3740 1 -- start start 2024-09-05T01:20:20.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebd9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebda540 0x55922ebdca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55922eb382f0 con 0x55922ebd54b0 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55922eae0270 con 0x55922ebda540 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0761be3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55922eb342b0 con 0x55922ebd4b70 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57642/0 (socket says 172.21.15.129:57642) 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 -- 172.21.15.129:0/1708730482 learned_addr learned my addr 172.21.15.129:0/1708730482 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebda540 0x55922ebdca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 -- 172.21.15.129:0/1708730482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebda540 msgr2=0x55922ebdca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebda540 0x55922ebdca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 -- 172.21.15.129:0/1708730482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 msgr2=0x55922ebd9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebd9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.248+0000 7f0757fff640 1 -- 172.21.15.129:0/1708730482 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922eadf340 con 0x55922ebd4b70 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.249+0000 7f0757fff640 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0750009460 tx=0x7f0750009670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20bf1f98446a8ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.249+0000 7f074effd640 1 -- 172.21.15.129:0/1708730482 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750004030 con 0x55922ebd4b70 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.249+0000 7f0755ffb640 1 -- 172.21.15.129:0/74156990 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0748042070 con 0x7f07581054b0 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.249+0000 7f074effd640 1 -- 172.21.15.129:0/1708730482 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f075000fc10 con 0x55922ebd4b70 2024-09-05T01:20:20.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.249+0000 7f074effd640 1 -- 172.21.15.129:0/1708730482 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750003850 con 0x55922ebd4b70 2024-09-05T01:20:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0755ffb640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 0x7f07440a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0755ffb640 1 -- 172.21.15.129:0/74156990 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f07480efc40 con 0x7f07581054b0 2024-09-05T01:20:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0761be3740 1 -- 172.21.15.129:0/1708730482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 msgr2=0x55922ebd4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0761be3740 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0750009460 tx=0x7f0750009670 comp rx=0 tx=0).stop 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0761be3740 1 -- 172.21.15.129:0/1708730482 shutdown_connections 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.250+0000 7f0761be3740 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebda540 0x55922ebdca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f075cb09640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 0x7f07440a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f0761be3740 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebd9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f0761be3740 1 --2- 172.21.15.129:0/1708730482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd4b70 0x55922ebd4f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f0761be3740 1 -- 172.21.15.129:0/1708730482 >> 172.21.15.129:0/1708730482 conn(0x55922ebd01e0 msgr2=0x55922ebd05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f0761be3740 1 -- 172.21.15.129:0/1708730482 shutdown_connections 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.251+0000 7f0761be3740 1 -- 172.21.15.129:0/1708730482 wait complete. 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.252+0000 7f0761be3740 1 -- start start 2024-09-05T01:20:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.252+0000 7f0761be3740 1 Processor -- start 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.253+0000 7f0761be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebefdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.253+0000 7f0761be3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 0x55922ebf02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f0757fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 0x55922ebf02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f0757fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 0x55922ebf02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57654/0 (socket says 172.21.15.129:57654) 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f0757fff640 1 -- 172.21.15.129:0/2280637500 learned_addr learned my addr 172.21.15.129:0/2280637500 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 0x55922ebf72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f075d30a640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 0x55922ebf72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.254+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55922ebee510 con 0x55922ebda540 2024-09-05T01:20:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55922eac1290 con 0x55922ebd4b70 2024-09-05T01:20:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55922ebee690 con 0x55922ebd54b0 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f0761be3740 10 client.0 _ll_register_callbacks cb 0x55922eaf2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075cb09640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebefdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075cb09640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 0x7f07440a5140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55922ebd1060 tx=0x7f0740004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075d30a640 1 -- 172.21.15.129:0/2280637500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 msgr2=0x55922ebf02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075d30a640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 0x55922ebf02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075d30a640 1 -- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 msgr2=0x55922ebefdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075d30a640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebefdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.255+0000 7f075d30a640 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922e84b210 con 0x55922ebda540 2024-09-05T01:20:20.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.256+0000 7f075d30a640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 0x55922ebf72c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07480b8490 tx=0x7f07480bba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f730a1f656f3c472 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.256+0000 7f0737fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:20:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.256+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748049040 con 0x55922ebda540 2024-09-05T01:20:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.256+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f07480c00d0 con 0x55922ebda540 2024-09-05T01:20:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.256+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f075000fc10 con 0x55922ebda540 2024-09-05T01:20:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.257+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07480bc030 con 0x55922ebda540 2024-09-05T01:20:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.257+0000 7f0761be3740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.257+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55922e917620 con 0x55922ebda540 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.257+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f075000fc10 con 0x55922ebda540 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.257+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f07480bc290 con 0x55922ebda540 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 20 client.15873 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ebfbc00 0x55922ebfe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 4 client.15873 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55922eb45a20 con 0x55922ebfbc00 2024-09-05T01:20:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.258+0000 7f075cb09640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ebfbc00 0x55922ebfe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.259+0000 7f075cb09640 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ebfbc00 0x55922ebfe060 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9049f33b0cd9df9a server_cookie=79806e6657a02201 in_seq=0 out_seq=0 2024-09-05T01:20:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.259+0000 7f0737fff640 10 client.15873 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0737fff640 1 -- 172.21.15.129:0/2280637500 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55922eb45a20 con 0x55922ebfbc00 2024-09-05T01:20:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0737fff640 10 client.15873 handle_command_reply: tid=0 2024-09-05T01:20:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 client.15873 shutdown 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 msgr2=0x55922ebf72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 0x55922ebf72c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f07480b8490 tx=0x7f07480bba50 comp rx=0 tx=0).stop 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 shutdown_connections 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ebfbc00 0x55922ebfe060 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebda540 0x55922ebf72c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebd54b0 0x55922ebf02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.260+0000 7f0761be3740 1 --2- 172.21.15.129:0/2280637500 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebefdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.261+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 >> 172.21.15.129:0/2280637500 conn(0x55922ebd01e0 msgr2=0x55922ebd19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.261+0000 7f0737fff640 0 client.15873 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.261+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 shutdown_connections 2024-09-05T01:20:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.261+0000 7f0761be3740 1 -- 172.21.15.129:0/2280637500 wait complete. 2024-09-05T01:20:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.262+0000 7f0761be3740 20 client.15873 trim_cache size 0 max 16384 2024-09-05T01:20:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.292+0000 7f0761be3740 1 Processor -- start 2024-09-05T01:20:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.293+0000 7f0761be3740 1 -- start start 2024-09-05T01:20:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd4b70 0x55922ebcea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd54b0 0x55922ebf7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07581aad00 con 0x55922ebd4b70 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a85d0 con 0x55922ebd54b0 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0761be3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0758074180 con 0x55922ebda540 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f075d30a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f075d30a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57670/0 (socket says 172.21.15.129:57670) 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f075d30a640 1 -- 172.21.15.129:0/1251022600 learned_addr learned my addr 172.21.15.129:0/1251022600 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.294+0000 7f0757fff640 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd54b0 0x55922ebf7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 -- 172.21.15.129:0/1251022600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd54b0 msgr2=0x55922ebf7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd54b0 0x55922ebf7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 -- 172.21.15.129:0/1251022600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd4b70 msgr2=0x55922ebcea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd4b70 0x55922ebcea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 -- 172.21.15.129:0/1251022600 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55922ebd47b0 con 0x55922ebda540 2024-09-05T01:20:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f075d30a640 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f07480c0040 tx=0x7f0748009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2d91252a75aa24 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.295+0000 7f074e7fc640 1 -- 172.21.15.129:0/1251022600 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748049070 con 0x55922ebda540 2024-09-05T01:20:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.296+0000 7f074e7fc640 1 -- 172.21.15.129:0/1251022600 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07480c4ca0 con 0x55922ebda540 2024-09-05T01:20:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.296+0000 7f074e7fc640 1 -- 172.21.15.129:0/1251022600 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074803b460 con 0x55922ebda540 2024-09-05T01:20:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 -- 172.21.15.129:0/1251022600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 msgr2=0x55922ebcefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f07480c0040 tx=0x7f0748009260 comp rx=0 tx=0).stop 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 -- 172.21.15.129:0/1251022600 shutdown_connections 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ebcefd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd54b0 0x55922ebf7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 --2- 172.21.15.129:0/1251022600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd4b70 0x55922ebcea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 -- 172.21.15.129:0/1251022600 >> 172.21.15.129:0/1251022600 conn(0x55922ebd01e0 msgr2=0x55922ebefdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 -- 172.21.15.129:0/1251022600 shutdown_connections 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.297+0000 7f0761be3740 1 -- 172.21.15.129:0/1251022600 wait complete. 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 -- start start 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 Processor -- start 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebf4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ec64930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a6a90 con 0x55922ebd54b0 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a7f00 con 0x55922ebd4b70 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.298+0000 7f0761be3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f07581a7160 con 0x55922ebda540 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f0761be3740 10 client.0 _ll_register_callbacks cb 0x7f0748009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56870/0 (socket says 172.21.15.129:56870) 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 -- 172.21.15.129:0/4130467873 learned_addr learned my addr 172.21.15.129:0/4130467873 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f0757fff640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebf4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 -- 172.21.15.129:0/4130467873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 msgr2=0x55922ec64930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ec64930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 -- 172.21.15.129:0/4130467873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 msgr2=0x55922ebf4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebf4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.299+0000 7f075cb09640 1 -- 172.21.15.129:0/4130467873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55922ec64ea0 con 0x55922ebd4b70 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.300+0000 7f075cb09640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55922ebfe910 tx=0x7f0740004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf8291d92c7a19c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.300+0000 7f0737fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.300+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074002c060 con 0x55922ebd4b70 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.300+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f074001e9b0 con 0x55922ebd4b70 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.300+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07480c4ca0 con 0x55922ebd4b70 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f074001ebb0 con 0x55922ebd4b70 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 10 client.25421 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55922ec66280 con 0x55922ebd4b70 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07480c4ca0 con 0x55922ebd4b70 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f0740026030 con 0x55922ebd4b70 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 10 client.25421 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 20 client.25421 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ec66ae0 0x55922ec68f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 4 client.25421 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.301+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55922eb3fd70 con 0x55922ec66ae0 2024-09-05T01:20:20.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.302+0000 7f0757fff640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ec66ae0 0x55922ec68f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.303+0000 7f0757fff640 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ec66ae0 0x55922ec68f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78f28cd0942d2a13 server_cookie=f0e9cf7948c4eee0 in_seq=0 out_seq=0 2024-09-05T01:20:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.303+0000 7f0737fff640 10 client.25421 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.328+0000 7f0737fff640 1 -- 172.21.15.129:0/4130467873 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55922eb3fd70 con 0x55922ec66ae0 2024-09-05T01:20:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.328+0000 7f0737fff640 10 client.25421 handle_command_reply: tid=0 2024-09-05T01:20:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.328+0000 7f0761be3740 1 client.25421 shutdown 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 msgr2=0x55922ebf4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55922ebfe910 tx=0x7f0740004ab0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 shutdown_connections 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55922ec66ae0 0x55922ec68f00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55922ebda540 0x55922ec64930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55922ebd54b0 0x55922ebf4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 --2- 172.21.15.129:0/4130467873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55922ebd4b70 0x55922ebf4230 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 >> 172.21.15.129:0/4130467873 conn(0x55922ebd01e0 msgr2=0x55922ebd1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.330+0000 7f0737fff640 0 client.25421 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.331+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 shutdown_connections 2024-09-05T01:20:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.331+0000 7f0761be3740 1 -- 172.21.15.129:0/4130467873 wait complete. 2024-09-05T01:20:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.331+0000 7f0761be3740 20 client.25421 trim_cache size 0 max 16384 2024-09-05T01:20:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:20:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:20:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:20:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 msgr2=0x7f07440a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 0x7f07440a5140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55922ebd1060 tx=0x7f0740004a40 comp rx=0 tx=0).stop 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 msgr2=0x7f07581a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0748005ab0 tx=0x7f0748003b90 comp rx=0 tx=0).stop 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 shutdown_connections 2024-09-05T01:20:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f07440a2c80 0x7f07440a5140 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.332+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f07581054b0 0x7f07581a5720 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.333+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0758102b00 0x7f075819e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.333+0000 7f075db0b640 1 --2- 172.21.15.129:0/74156990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f07581001b0 0x7f075819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.333+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 >> 172.21.15.129:0/74156990 conn(0x7f07580fbfa0 msgr2=0x7f0758100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.333+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 shutdown_connections 2024-09-05T01:20:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:20.333+0000 7f075db0b640 1 -- 172.21.15.129:0/74156990 wait complete. 2024-09-05T01:20:21.244 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:20 smithi129 ceph-mon[30979]: pgmap v502: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-09-05T01:20:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:20 smithi194 ceph-mon[34941]: pgmap v502: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-09-05T01:20:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:20 smithi183 ceph-mon[34589]: pgmap v502: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.6 MiB/s wr, 1.05k op/s 2024-09-05T01:20:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:22 smithi129 ceph-mon[30979]: pgmap v503: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 917 op/s 2024-09-05T01:20:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:22 smithi194 ceph-mon[34941]: pgmap v503: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 917 op/s 2024-09-05T01:20:23.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:22 smithi183 ceph-mon[34589]: pgmap v503: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 MiB/s wr, 917 op/s 2024-09-05T01:20:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 200.0 s, 955.6 tps, lat 103.401 ms stddev 45.482 2024-09-05T01:20:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:24 smithi129 ceph-mon[30979]: pgmap v504: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:24 smithi194 ceph-mon[34941]: pgmap v504: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:24 smithi183 ceph-mon[34589]: pgmap v504: 129 pgs: 129 active+clean; 1003 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.6 MiB/s wr, 1.14k op/s 2024-09-05T01:20:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:26 smithi129 ceph-mon[30979]: pgmap v505: 129 pgs: 129 active+clean; 1003 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-05T01:20:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:26 smithi194 ceph-mon[34941]: pgmap v505: 129 pgs: 129 active+clean; 1003 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-05T01:20:27.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:26 smithi183 ceph-mon[34589]: pgmap v505: 129 pgs: 129 active+clean; 1003 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.8 MiB/s wr, 1.03k op/s 2024-09-05T01:20:29.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:28 smithi129 ceph-mon[30979]: pgmap v506: 129 pgs: 129 active+clean; 1003 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-09-05T01:20:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:28 smithi194 ceph-mon[34941]: pgmap v506: 129 pgs: 129 active+clean; 1003 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-09-05T01:20:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:28 smithi183 ceph-mon[34589]: pgmap v506: 129 pgs: 129 active+clean; 1003 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 9.0 MiB/s wr, 1.08k op/s 2024-09-05T01:20:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 205.0 s, 999.2 tps, lat 101.732 ms stddev 32.529 2024-09-05T01:20:31.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:30 smithi183 ceph-mon[34589]: pgmap v507: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-09-05T01:20:31.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:30 smithi129 ceph-mon[30979]: pgmap v507: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-09-05T01:20:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:30 smithi194 ceph-mon[34941]: pgmap v507: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 MiB/s wr, 1.00k op/s 2024-09-05T01:20:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:32 smithi129 ceph-mon[30979]: pgmap v508: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 922 op/s 2024-09-05T01:20:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:32 smithi194 ceph-mon[34941]: pgmap v508: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 922 op/s 2024-09-05T01:20:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:32 smithi183 ceph-mon[34589]: pgmap v508: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.8 MiB/s wr, 922 op/s 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 -- 172.21.15.129:0/3170495842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc1001b0 msgr2=0x7febdc1025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 --2- 172.21.15.129:0/3170495842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc1001b0 0x7febdc1025c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7febcc009900 tx=0x7febcc02a420 comp rx=0 tx=0).stop 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 -- 172.21.15.129:0/3170495842 shutdown_connections 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 --2- 172.21.15.129:0/3170495842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 0x7febdc105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 --2- 172.21.15.129:0/3170495842 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc102b00 0x7febdc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 --2- 172.21.15.129:0/3170495842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc1001b0 0x7febdc1025c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.488+0000 7febdbfff640 1 -- 172.21.15.129:0/3170495842 >> 172.21.15.129:0/3170495842 conn(0x7febdc0fbfa0 msgr2=0x7febdc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.489+0000 7febdbfff640 1 -- 172.21.15.129:0/3170495842 shutdown_connections 2024-09-05T01:20:33.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.490+0000 7febdbfff640 1 -- 172.21.15.129:0/3170495842 wait complete. 2024-09-05T01:20:33.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 Processor -- start 2024-09-05T01:20:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 -- start start 2024-09-05T01:20:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc1001b0 0x7febdc19e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 0x7febdc1a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.492 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febdc113b30 con 0x7febdc1054b0 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febdc1139b0 con 0x7febdc102b00 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febdbfff640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febdc113cb0 con 0x7febdc1001b0 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febda7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.491+0000 7febda7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56958/0 (socket says 172.21.15.129:56958) 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.492+0000 7febda7fc640 1 -- 172.21.15.129:0/4081860944 learned_addr learned my addr 172.21.15.129:0/4081860944 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:33.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.492+0000 7febdb7fe640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 0x7febdc1a57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 -- 172.21.15.129:0/4081860944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc1001b0 msgr2=0x7febdc19e220 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc1001b0 0x7febdc19e220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 -- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 msgr2=0x7febdc1a57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 0x7febdc1a57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 -- 172.21.15.129:0/4081860944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc06bd40 con 0x7febdc102b00 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febda7fc640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7febd000b450 tx=0x7febd000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febc3fff640 1 -- 172.21.15.129:0/4081860944 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0015020 con 0x7febdc102b00 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc06c030 con 0x7febdc102b00 2024-09-05T01:20:33.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.493+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc06c600 con 0x7febdc102b00 2024-09-05T01:20:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.494+0000 7febc3fff640 1 -- 172.21.15.129:0/4081860944 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febd0011d40 con 0x7febdc102b00 2024-09-05T01:20:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.494+0000 7febc3fff640 1 -- 172.21.15.129:0/4081860944 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd000f340 con 0x7febdc102b00 2024-09-05T01:20:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 Processor -- start 2024-09-05T01:20:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 -- start start 2024-09-05T01:20:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e540f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e548a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56498e4a42f0 con 0x56498e546540 2024-09-05T01:20:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56498e44c270 con 0x56498e5414b0 2024-09-05T01:20:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.497+0000 7febe45fa740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56498e4a02b0 con 0x56498e540b70 2024-09-05T01:20:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56970/0 (socket says 172.21.15.129:56970) 2024-09-05T01:20:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 -- 172.21.15.129:0/2830611260 learned_addr learned my addr 172.21.15.129:0/2830611260 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:33.499 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdb7fe640 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e540f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 -- 172.21.15.129:0/2830611260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 msgr2=0x56498e540f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e540f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 -- 172.21.15.129:0/2830611260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 msgr2=0x56498e548a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e548a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febdaffd640 1 -- 172.21.15.129:0/2830611260 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498e44b340 con 0x56498e5414b0 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.498+0000 7febc3fff640 1 -- 172.21.15.129:0/4081860944 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7febd0018020 con 0x7febdc102b00 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.499+0000 7febdaffd640 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7febcc0314e0 tx=0x7febcc0040b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cadfe03db479aadd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.499+0000 7febc17fa640 1 -- 172.21.15.129:0/2830611260 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc03e070 con 0x56498e5414b0 2024-09-05T01:20:33.500 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.499+0000 7febc17fa640 1 -- 172.21.15.129:0/2830611260 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febcc0054e0 con 0x56498e5414b0 2024-09-05T01:20:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.499+0000 7febc17fa640 1 -- 172.21.15.129:0/2830611260 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc035040 con 0x56498e5414b0 2024-09-05T01:20:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febc3fff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 0x7febc40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febc3fff640 1 -- 172.21.15.129:0/4081860944 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7febd00ceec0 con 0x7febdc102b00 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 -- 172.21.15.129:0/2830611260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 msgr2=0x56498e545f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7febcc0314e0 tx=0x7febcc0040b0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febdb7fe640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 0x7febc40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 -- 172.21.15.129:0/2830611260 shutdown_connections 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e548a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e545f70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 --2- 172.21.15.129:0/2830611260 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e540f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.500+0000 7febe45fa740 1 -- 172.21.15.129:0/2830611260 >> 172.21.15.129:0/2830611260 conn(0x56498e53c1e0 msgr2=0x56498e53c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.501+0000 7febe45fa740 1 -- 172.21.15.129:0/2830611260 shutdown_connections 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.501+0000 7febe45fa740 1 -- 172.21.15.129:0/2830611260 wait complete. 2024-09-05T01:20:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.501+0000 7febe45fa740 1 -- start start 2024-09-05T01:20:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.501+0000 7febe45fa740 1 Processor -- start 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.502+0000 7febdb7fe640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 0x7febc40a5100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febd4005fd0 tx=0x7febd4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.502+0000 7febe45fa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 0x56498e54adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.502+0000 7febe45fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.502+0000 7febe45fa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e546540 0x56498e552330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febda7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febda7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56986/0 (socket says 172.21.15.129:56986) 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febda7fc640 1 -- 172.21.15.129:0/163952628 learned_addr learned my addr 172.21.15.129:0/163952628 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56498e549500 con 0x56498e540b70 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56498e42d290 con 0x56498e5414b0 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56498e549680 con 0x56498e546540 2024-09-05T01:20:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febdaffd640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 0x56498e54adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.503+0000 7febe45fa740 10 client.0 _ll_register_callbacks cb 0x56498e45e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 -- 172.21.15.129:0/163952628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e546540 msgr2=0x56498e552330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e546540 0x56498e552330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 -- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 msgr2=0x56498e54b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498e1b7210 con 0x56498e540b70 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febdaffd640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 0x56498e54adb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7febcc009900 tx=0x7febcc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57daab40cad92481 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febcc0054e0 con 0x56498e540b70 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febbeff5640 1 -- 172.21.15.129:0/163952628 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc03e030 con 0x56498e540b70 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.504+0000 7febbeff5640 1 -- 172.21.15.129:0/163952628 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7febcc004ca0 con 0x56498e540b70 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.505+0000 7febbeff5640 1 -- 172.21.15.129:0/163952628 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febcc053050 con 0x56498e540b70 2024-09-05T01:20:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.505+0000 7febe45fa740 10 client.15885 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.505+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=30}) -- 0x56498e283620 con 0x56498e540b70 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febbeff5640 1 -- 172.21.15.129:0/163952628 <== mon.0 v2:172.21.15.129:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7febcc004ea0 con 0x56498e540b70 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 10 client.15885 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 20 client.15885 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e558e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 4 client.15885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7febcc0054e0 con 0x56498e556940 2024-09-05T01:20:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.507+0000 7febda7fc640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e558e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.509+0000 7febda7fc640 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e558e00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09ea35c5b76a2a9 server_cookie=376c45df6d0e4d41 in_seq=0 out_seq=0 2024-09-05T01:20:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.509+0000 7febbeff5640 10 client.15885 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.509+0000 7febbeff5640 1 -- 172.21.15.129:0/163952628 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7febd0080e80 con 0x56498e556940 2024-09-05T01:20:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.509+0000 7febbeff5640 10 client.15885 handle_command_reply: tid=0 2024-09-05T01:20:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 client.15885 shutdown 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 msgr2=0x56498e54adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 0x56498e54adb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7febcc009900 tx=0x7febcc0048f0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 shutdown_connections 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e558e00 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e546540 0x56498e552330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febbeff5640 0 client.15885 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 --2- 172.21.15.129:0/163952628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e540b70 0x56498e54adb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.510+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 >> 172.21.15.129:0/163952628 conn(0x56498e53c1e0 msgr2=0x56498e53d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.513+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 shutdown_connections 2024-09-05T01:20:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.513+0000 7febe45fa740 1 -- 172.21.15.129:0/163952628 wait complete. 2024-09-05T01:20:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.513+0000 7febe45fa740 20 client.15885 trim_cache size 0 max 16384 2024-09-05T01:20:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.550+0000 7febe45fa740 1 Processor -- start 2024-09-05T01:20:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.550+0000 7febe45fa740 1 -- start start 2024-09-05T01:20:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febe45fa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febe45fa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e5414b0 0x56498e5522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febdaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febdaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:41920/0 (socket says 172.21.15.129:41920) 2024-09-05T01:20:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febdaffd640 1 -- 172.21.15.129:0/3326574806 learned_addr learned my addr 172.21.15.129:0/3326574806 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febe45fa740 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e546540 0x56498e53b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febda7fc640 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e5414b0 0x56498e5522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.551+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febc80126c0 con 0x56498e5414b0 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febc800ff90 con 0x56498e546540 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febc800dde0 con 0x56498e540b70 2024-09-05T01:20:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdb7fe640 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e546540 0x56498e53b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdaffd640 1 -- 172.21.15.129:0/3326574806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e546540 msgr2=0x56498e53b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdaffd640 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e546540 0x56498e53b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdaffd640 1 -- 172.21.15.129:0/3326574806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e5414b0 msgr2=0x56498e5522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdaffd640 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e5414b0 0x56498e5522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.552+0000 7febdaffd640 1 -- 172.21.15.129:0/3326574806 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498e53c000 con 0x56498e540b70 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.553+0000 7febdaffd640 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febcc005320 tx=0x7febcc047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37b643225f3e575 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.553+0000 7febc0ff9640 1 -- 172.21.15.129:0/3326574806 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc05d070 con 0x56498e540b70 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.553+0000 7febc0ff9640 1 -- 172.21.15.129:0/3326574806 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febcc004f80 con 0x56498e540b70 2024-09-05T01:20:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.553+0000 7febc0ff9640 1 -- 172.21.15.129:0/3326574806 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc03e070 con 0x56498e540b70 2024-09-05T01:20:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.554+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 msgr2=0x56498e54b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febcc005320 tx=0x7febcc047000 comp rx=0 tx=0).stop 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 shutdown_connections 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e546540 0x56498e53b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e5414b0 0x56498e5522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 --2- 172.21.15.129:0/3326574806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54b380 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 >> 172.21.15.129:0/3326574806 conn(0x56498e53c1e0 msgr2=0x56498e53aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 shutdown_connections 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.555+0000 7febe45fa740 1 -- 172.21.15.129:0/3326574806 wait complete. 2024-09-05T01:20:33.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 -- start start 2024-09-05T01:20:33.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 Processor -- start 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e5bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7febc800e450 con 0x56498e546540 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febdaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febe45fa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7febc800f8c0 con 0x56498e5414b0 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febdaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:41926/0 (socket says 172.21.15.129:41926) 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febdaffd640 1 -- 172.21.15.129:0/1727162902 learned_addr learned my addr 172.21.15.129:0/1727162902 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.556+0000 7febda7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7febc800eb20 con 0x56498e540b70 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febdb7fe640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e5bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febe45fa740 10 client.0 _ll_register_callbacks cb 0x7febcc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febda7fc640 1 -- 172.21.15.129:0/1727162902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 msgr2=0x56498e54f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febda7fc640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febda7fc640 1 -- 172.21.15.129:0/1727162902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 msgr2=0x56498e5bf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febda7fc640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e5bf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.557+0000 7febda7fc640 1 -- 172.21.15.129:0/1727162902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498e5bfbd0 con 0x56498e5414b0 2024-09-05T01:20:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.558+0000 7febda7fc640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54f6b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febd009b2a0 tx=0x7febd009b360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed834228d2cfe09e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.558+0000 7febbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:20:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.559+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd0099610 con 0x56498e5414b0 2024-09-05T01:20:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.559+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7febd0097980 con 0x56498e5414b0 2024-09-05T01:20:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.559+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febcc004f80 con 0x56498e5414b0 2024-09-05T01:20:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.560+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7febd00a3af0 con 0x56498e5414b0 2024-09-05T01:20:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.560+0000 7febe45fa740 10 client.25439 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.560+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56498e5c0e70 con 0x56498e5414b0 2024-09-05T01:20:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.561+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febcc004f80 con 0x56498e5414b0 2024-09-05T01:20:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.561+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7febd00bb0b0 con 0x56498e5414b0 2024-09-05T01:20:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 10 client.25439 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 10 client.25439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 20 client.25439 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e5c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 4 client.25439 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:20:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.562+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56498e4b1a20 con 0x56498e556940 2024-09-05T01:20:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.563+0000 7febdaffd640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e5c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.564+0000 7febbeff5640 10 client.25439 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:33.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.564+0000 7febdaffd640 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e5c37f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d496c6049e5102 server_cookie=c7164d51f606ac68 in_seq=0 out_seq=0 2024-09-05T01:20:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.585+0000 7febbeff5640 1 -- 172.21.15.129:0/1727162902 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56498e4b1a20 con 0x56498e556940 2024-09-05T01:20:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.585+0000 7febbeff5640 10 client.25439 handle_command_reply: tid=0 2024-09-05T01:20:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.586+0000 7febe45fa740 1 client.25439 shutdown 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 msgr2=0x56498e54f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54f6b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7febd009b2a0 tx=0x7febd009b360 comp rx=0 tx=0).stop 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 shutdown_connections 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56498e556940 0x56498e5c37f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56498e546540 0x56498e5bf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56498e5414b0 0x56498e54f6b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 --2- 172.21.15.129:0/1727162902 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56498e540b70 0x56498e54f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 >> 172.21.15.129:0/1727162902 conn(0x56498e53c1e0 msgr2=0x56498e53aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febbeff5640 0 client.25439 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 shutdown_connections 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.588+0000 7febe45fa740 1 -- 172.21.15.129:0/1727162902 wait complete. 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.589+0000 7febe45fa740 20 client.25439 trim_cache size 0 max 16384 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:20:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:20:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:20:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:20:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.593+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 msgr2=0x7febc40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.593+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 0x7febc40a5100 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7febd4005fd0 tx=0x7febd4009500 comp rx=0 tx=0).stop 2024-09-05T01:20:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.593+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 msgr2=0x7febdc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.593+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7febd000b450 tx=0x7febd000b910 comp rx=0 tx=0).stop 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 shutdown_connections 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7febc40a2c40 0x7febc40a5100 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7febdc1054b0 0x7febdc1a57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7febdc102b00 0x7febdc19e760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 --2- 172.21.15.129:0/4081860944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7febdc1001b0 0x7febdc19e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.594+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 >> 172.21.15.129:0/4081860944 conn(0x7febdc0fbfa0 msgr2=0x7febdc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.596+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 shutdown_connections 2024-09-05T01:20:33.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:33.596+0000 7febdbfff640 1 -- 172.21.15.129:0/4081860944 wait complete. 2024-09-05T01:20:35.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 210.0 s, 1049.4 tps, lat 95.173 ms stddev 29.046 2024-09-05T01:20:35.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:34 smithi129 ceph-mon[30979]: pgmap v509: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:35.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:34 smithi194 ceph-mon[34941]: pgmap v509: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:35.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:34 smithi183 ceph-mon[34589]: pgmap v509: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.3 MiB/s wr, 1.10k op/s 2024-09-05T01:20:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:36 smithi129 ceph-mon[30979]: pgmap v510: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 954 op/s 2024-09-05T01:20:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:36 smithi194 ceph-mon[34941]: pgmap v510: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 954 op/s 2024-09-05T01:20:37.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:36 smithi183 ceph-mon[34589]: pgmap v510: 129 pgs: 129 active+clean; 1003 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.0 MiB/s wr, 954 op/s 2024-09-05T01:20:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:38 smithi129 ceph-mon[30979]: pgmap v511: 129 pgs: 129 active+clean; 1003 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-05T01:20:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:38 smithi194 ceph-mon[34941]: pgmap v511: 129 pgs: 129 active+clean; 1003 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-05T01:20:39.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:38 smithi183 ceph-mon[34589]: pgmap v511: 129 pgs: 129 active+clean; 1003 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 MiB/s wr, 1.13k op/s 2024-09-05T01:20:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 215.0 s, 1044.4 tps, lat 95.619 ms stddev 64.310 2024-09-05T01:20:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:40 smithi129 ceph-mon[30979]: pgmap v512: 129 pgs: 129 active+clean; 1004 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 1.07k op/s 2024-09-05T01:20:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:40 smithi194 ceph-mon[34941]: pgmap v512: 129 pgs: 129 active+clean; 1004 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 1.07k op/s 2024-09-05T01:20:41.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:40 smithi183 ceph-mon[34589]: pgmap v512: 129 pgs: 129 active+clean; 1004 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.1 MiB/s wr, 1.07k op/s 2024-09-05T01:20:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:42 smithi129 ceph-mon[30979]: pgmap v513: 129 pgs: 129 active+clean; 1004 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 956 op/s 2024-09-05T01:20:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:42 smithi194 ceph-mon[34941]: pgmap v513: 129 pgs: 129 active+clean; 1004 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 956 op/s 2024-09-05T01:20:43.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:42 smithi183 ceph-mon[34589]: pgmap v513: 129 pgs: 129 active+clean; 1004 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 8.0 MiB/s wr, 956 op/s 2024-09-05T01:20:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 220.0 s, 1049.0 tps, lat 95.613 ms stddev 103.500 2024-09-05T01:20:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:44 smithi129 ceph-mon[30979]: pgmap v514: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-05T01:20:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:44 smithi194 ceph-mon[34941]: pgmap v514: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-05T01:20:45.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:44 smithi183 ceph-mon[34589]: pgmap v514: 129 pgs: 129 active+clean; 1019 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 11 MiB/s wr, 1.19k op/s 2024-09-05T01:20:46.244 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.3 2024-09-05T01:20:46.245 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:20:46.707 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:20:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 -- 172.21.15.129:0/1130420739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 msgr2=0x7feddc108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 --2- 172.21.15.129:0/1130420739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc108870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009900 tx=0x7fedd002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 -- 172.21.15.129:0/1130420739 shutdown_connections 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 --2- 172.21.15.129:0/1130420739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feddc109d40 0x7feddc1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 --2- 172.21.15.129:0/1130420739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109380 0x7feddc109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 --2- 172.21.15.129:0/1130420739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc108870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 -- 172.21.15.129:0/1130420739 >> 172.21.15.129:0/1130420739 conn(0x7feddc077490 msgr2=0x7feddc0798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 -- 172.21.15.129:0/1130420739 shutdown_connections 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.776+0000 7fede2e53640 1 -- 172.21.15.129:0/1130420739 wait complete. 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 Processor -- start 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 -- start start 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc07f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feddc109380 0x7feddc07feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feddc113d50 con 0x7feddc109380 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feddc113bd0 con 0x7feddc108470 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.777+0000 7fede2e53640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feddc113ed0 con 0x7feddc109d40 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede0bef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc07f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44888/0 (socket says 172.21.15.129:44888) 2024-09-05T01:20:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 -- 172.21.15.129:0/374543978 learned_addr learned my addr 172.21.15.129:0/374543978 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 -- 172.21.15.129:0/374543978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 msgr2=0x7feddc07f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc07f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 -- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feddc109380 msgr2=0x7feddc07feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feddc109380 0x7feddc07feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 -- 172.21.15.129:0/374543978 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feddc080550 con 0x7feddc109d40 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.778+0000 7fede13f0640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b970 tx=0x7fedd400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.779+0000 7fedd9ffb640 1 -- 172.21.15.129:0/374543978 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4019070 con 0x7feddc109d40 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.779+0000 7fedd9ffb640 1 -- 172.21.15.129:0/374543978 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd4014440 con 0x7feddc109d40 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.779+0000 7fedd9ffb640 1 -- 172.21.15.129:0/374543978 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40075b0 con 0x7feddc109d40 2024-09-05T01:20:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.779+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feddc1aeac0 con 0x7feddc109d40 2024-09-05T01:20:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.780+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feddc1af010 con 0x7feddc109d40 2024-09-05T01:20:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.783+0000 7fedd9ffb640 1 -- 172.21.15.129:0/374543978 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fedd401c070 con 0x7feddc109d40 2024-09-05T01:20:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.785+0000 7fede5bf1740 1 Processor -- start 2024-09-05T01:20:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.785+0000 7fedd9ffb640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 0x7fedc40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fedd9ffb640 1 -- 172.21.15.129:0/374543978 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fedd40cebc0 con 0x7feddc109d40 2024-09-05T01:20:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 -- start start 2024-09-05T01:20:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7feddbfff640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 0x7fedc40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd829a0 0x559fedd82da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd88370 0x559fedd8a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559fedce62f0 con 0x559fedd829a0 2024-09-05T01:20:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fedc8e270 con 0x559fedd88370 2024-09-05T01:20:46.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.786+0000 7fede5bf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559fedce22b0 con 0x559fedd832e0 2024-09-05T01:20:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7feddbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7feddbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44898/0 (socket says 172.21.15.129:44898) 2024-09-05T01:20:46.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7feddbfff640 1 -- 172.21.15.129:0/1624428476 learned_addr learned my addr 172.21.15.129:0/1624428476 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7fede13f0640 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd88370 0x559fedd8a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7feddbfff640 1 -- 172.21.15.129:0/1624428476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd88370 msgr2=0x559fedd8a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.787+0000 7feddbfff640 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd88370 0x559fedd8a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.788+0000 7feddbfff640 1 -- 172.21.15.129:0/1624428476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd829a0 msgr2=0x559fedd82da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.788+0000 7feddbfff640 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd829a0 0x559fedd82da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.788+0000 7feddbfff640 1 -- 172.21.15.129:0/1624428476 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fedc8d340 con 0x559fedd832e0 2024-09-05T01:20:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.788+0000 7feddbfff640 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00e650 tx=0x7fedcc018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e685c4d316841b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.788+0000 7feddbfff640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 0x7fedc40a52b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009f80 tx=0x7fedcc0127c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.789+0000 7fedc2ffd640 1 -- 172.21.15.129:0/1624428476 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc023070 con 0x559fedd832e0 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.789+0000 7fedc2ffd640 1 -- 172.21.15.129:0/1624428476 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedcc012230 con 0x559fedd832e0 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.789+0000 7fedc2ffd640 1 -- 172.21.15.129:0/1624428476 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc01e740 con 0x559fedd832e0 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.790+0000 7fede5bf1740 1 -- 172.21.15.129:0/1624428476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 msgr2=0x559fedd87da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.790+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00e650 tx=0x7fedcc018000 comp rx=0 tx=0).stop 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 -- 172.21.15.129:0/1624428476 shutdown_connections 2024-09-05T01:20:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd88370 0x559fedd8a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd832e0 0x559fedd87da0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1624428476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd829a0 0x559fedd82da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 -- 172.21.15.129:0/1624428476 >> 172.21.15.129:0/1624428476 conn(0x559fedd7e010 msgr2=0x559fedd7e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.791+0000 7fede5bf1740 1 -- 172.21.15.129:0/1624428476 shutdown_connections 2024-09-05T01:20:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.792+0000 7fede5bf1740 1 -- 172.21.15.129:0/1624428476 wait complete. 2024-09-05T01:20:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.792+0000 7fede5bf1740 1 -- start start 2024-09-05T01:20:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.792+0000 7fede5bf1740 1 Processor -- start 2024-09-05T01:20:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd8cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd8d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559fedd8b330 con 0x559fedd832e0 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559fedc6f290 con 0x559fedd829a0 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559fedd8b4b0 con 0x559fedd88370 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede5bf1740 10 client.0 _ll_register_callbacks cb 0x559fedca0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.793+0000 7fede13f0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44900/0 (socket says 172.21.15.129:44900) 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 -- 172.21.15.129:0/1029839004 learned_addr learned my addr 172.21.15.129:0/1029839004 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 -- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 msgr2=0x559fedd8cbe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd8cbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 -- 172.21.15.129:0/1029839004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 msgr2=0x559fedd8d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd8d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 -- 172.21.15.129:0/1029839004 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fed9f9210 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fede13f0640 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedd4099090 tx=0x7fedd4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71881d5cf860d095 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fedbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.794+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40b3020 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.795+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fedcf3a20 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.795+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fedd40a37f0 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.795+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedd409f3c0 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.797+0000 7fede5bf1740 10 client.25399 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.797+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559fedac5620 con 0x559fedd88370 2024-09-05T01:20:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559fedcf3a20 con 0x559fedd88370 2024-09-05T01:20:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fedd409a570 con 0x559fedd88370 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 10 client.25399 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 10 client.25399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 20 client.25399 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fedda9c90 0x559feddac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 4 client.25399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.798+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fedcedd70 con 0x559fedda9c90 2024-09-05T01:20:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.799+0000 7fede0bef640 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fedda9c90 0x559feddac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.799+0000 7fede0bef640 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fedda9c90 0x559feddac120 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2452f1b8f1af46e server_cookie=fd7d6b73f44141af in_seq=0 out_seq=0 2024-09-05T01:20:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fedbbfff640 10 client.25399 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fedbbfff640 1 -- 172.21.15.129:0/1029839004 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fedcedd70 con 0x559fedda9c90 2024-09-05T01:20:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fedbbfff640 10 client.25399 handle_command_reply: tid=0 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fede5bf1740 1 client.25399 shutdown 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 msgr2=0x559fedd94160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.800+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fedd4099090 tx=0x7fedd4004a40 comp rx=0 tx=0).stop 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 shutdown_connections 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fedda9c90 0x559feddac120 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd94160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd8d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1029839004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd8cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 >> 172.21.15.129:0/1029839004 conn(0x559fedd7e010 msgr2=0x559fedd7f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fedbbfff640 0 client.25399 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 shutdown_connections 2024-09-05T01:20:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 1 -- 172.21.15.129:0/1029839004 wait complete. 2024-09-05T01:20:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.801+0000 7fede5bf1740 20 client.25399 trim_cache size 0 max 16384 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 Processor -- start 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 -- start start 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd829a0 0x559fedd7d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd832e0 0x559fedd94120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1b37f0 con 0x559fedd88370 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1b10c0 con 0x559fedd832e0 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede5bf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feddc074080 con 0x559fedd829a0 2024-09-05T01:20:46.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede13f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede13f0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55750/0 (socket says 172.21.15.129:55750) 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.829+0000 7fede13f0640 1 -- 172.21.15.129:0/1965423151 learned_addr learned my addr 172.21.15.129:0/1965423151 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.830+0000 7fede13f0640 1 -- 172.21.15.129:0/1965423151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd829a0 msgr2=0x559fedd7d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.830+0000 7fede13f0640 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd829a0 0x559fedd7d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.830+0000 7fede13f0640 1 -- 172.21.15.129:0/1965423151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd832e0 msgr2=0x559fedd94120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.830+0000 7fede13f0640 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd832e0 0x559fedd94120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.830+0000 7fede13f0640 1 -- 172.21.15.129:0/1965423151 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fedd782f0 con 0x559fedd88370 2024-09-05T01:20:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.831+0000 7fede13f0640 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fedd409afa0 tx=0x7fedd4099520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6732070e6e4a020 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.832+0000 7fedc27fc640 1 -- 172.21.15.129:0/1965423151 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40b3070 con 0x559fedd88370 2024-09-05T01:20:46.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.832+0000 7fedc27fc640 1 -- 172.21.15.129:0/1965423151 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedd40a3c80 con 0x559fedd88370 2024-09-05T01:20:46.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.832+0000 7fedc27fc640 1 -- 172.21.15.129:0/1965423151 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd40a3a50 con 0x559fedd88370 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 -- 172.21.15.129:0/1965423151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 msgr2=0x559feddb0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fedd409afa0 tx=0x7fedd4099520 comp rx=0 tx=0).stop 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 -- 172.21.15.129:0/1965423151 shutdown_connections 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd88370 0x559feddb0a20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd832e0 0x559fedd94120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 --2- 172.21.15.129:0/1965423151 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd829a0 0x559fedd7d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.833+0000 7fede5bf1740 1 -- 172.21.15.129:0/1965423151 >> 172.21.15.129:0/1965423151 conn(0x559fedd7e010 msgr2=0x559fedd7c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.835+0000 7fede5bf1740 1 -- 172.21.15.129:0/1965423151 shutdown_connections 2024-09-05T01:20:46.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.835+0000 7fede5bf1740 1 -- 172.21.15.129:0/1965423151 wait complete. 2024-09-05T01:20:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.835+0000 7fede5bf1740 1 -- start start 2024-09-05T01:20:46.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.836+0000 7fede5bf1740 1 Processor -- start 2024-09-05T01:20:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.836+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede5bf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede0bef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede0bef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:57530/0 (socket says 172.21.15.129:57530) 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede0bef640 1 -- 172.21.15.129:0/3248460200 learned_addr learned my addr 172.21.15.129:0/3248460200 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1af580 con 0x559fedd832e0 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1b09f0 con 0x559fedd829a0 2024-09-05T01:20:46.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede13f0640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.837+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feddc1afc50 con 0x559fedd88370 2024-09-05T01:20:46.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede5bf1740 10 client.0 _ll_register_callbacks cb 0x7fedd40972c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 -- 172.21.15.129:0/3248460200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 msgr2=0x559fedd929a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd929a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 -- 172.21.15.129:0/3248460200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 msgr2=0x559fedd919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:46.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fedd92fb0 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede0bef640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fedd002a9e0 tx=0x7fedd0004330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b51e74acf09a0f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fedbbfff640 10 client.25448 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd40a3c80 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.838+0000 7fedbbfff640 1 -- 172.21.15.129:0/3248460200 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0035040 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fedbbfff640 1 -- 172.21.15.129:0/3248460200 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fedd00035f0 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fedbbfff640 1 -- 172.21.15.129:0/3248460200 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fedd0052050 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 10 client.25448 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=30}) -- 0x559fedd94290 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fedbbfff640 1 -- 172.21.15.129:0/3248460200 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fedd0003a00 con 0x559fedd829a0 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 10 client.25448 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:46.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 20 client.25448 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fede14e90 0x559fede172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 4 client.25448 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.839+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fedc6a900 con 0x559fede14e90 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.840+0000 7feddbfff640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fede14e90 0x559fede172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.840+0000 7feddbfff640 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fede14e90 0x559fede172b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd95d93c1b7de2b6 server_cookie=86ea0c33c3e83751 in_seq=0 out_seq=0 2024-09-05T01:20:46.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.840+0000 7fedbbfff640 10 client.25448 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:46.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.863+0000 7fedbbfff640 1 -- 172.21.15.129:0/3248460200 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fedc6a900 con 0x559fede14e90 2024-09-05T01:20:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.863+0000 7fedbbfff640 10 client.25448 handle_command_reply: tid=0 2024-09-05T01:20:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.864+0000 7fede5bf1740 1 client.25448 shutdown 2024-09-05T01:20:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.864+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 msgr2=0x559fedd914a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.864+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fedd002a9e0 tx=0x7fedd0004330 comp rx=0 tx=0).stop 2024-09-05T01:20:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 shutdown_connections 2024-09-05T01:20:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559fede14e90 0x559fede172b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559fedd88370 0x559fedd929a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559fedd832e0 0x559fedd919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 --2- 172.21.15.129:0/3248460200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559fedd829a0 0x559fedd914a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 >> 172.21.15.129:0/3248460200 conn(0x559fedd7e010 msgr2=0x559fedd7f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fedbbfff640 0 client.25448 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.865+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 shutdown_connections 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.866+0000 7fede5bf1740 1 -- 172.21.15.129:0/3248460200 wait complete. 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.867+0000 7fede5bf1740 20 client.25448 trim_cache size 0 max 16384 2024-09-05T01:20:46.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:20:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:20:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:20:46.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 msgr2=0x7fedc40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 0x7fedc40a52b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009f80 tx=0x7fedcc0127c0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 msgr2=0x7feddc1ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fedd400b970 tx=0x7fedd400be30 comp rx=0 tx=0).stop 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 shutdown_connections 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fedc40a2df0 0x7fedc40a52b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feddc109d40 0x7feddc1ae460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feddc109380 0x7feddc07feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 --2- 172.21.15.129:0/374543978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feddc108470 0x7feddc07f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 >> 172.21.15.129:0/374543978 conn(0x7feddc077490 msgr2=0x7feddc10c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 shutdown_connections 2024-09-05T01:20:46.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:46.868+0000 7fede2e53640 1 -- 172.21.15.129:0/374543978 wait complete. 2024-09-05T01:20:47.155 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:46 smithi129 ceph-mon[30979]: pgmap v515: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-05T01:20:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:46 smithi194 ceph-mon[34941]: pgmap v515: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-05T01:20:47.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.295+0000 7f7d448f4640 1 -- 172.21.15.129:0/3421753392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d401091f0 msgr2=0x7f7d401095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:47.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.295+0000 7f7d448f4640 1 --2- 172.21.15.129:0/3421753392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d401091f0 0x7f7d401095d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009900 tx=0x7f7d3002a4b0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.297+0000 7f7d448f4640 1 -- 172.21.15.129:0/3421753392 shutdown_connections 2024-09-05T01:20:47.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.297+0000 7f7d448f4640 1 --2- 172.21.15.129:0/3421753392 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d40114ab0 0x7f7d40116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.297+0000 7f7d448f4640 1 --2- 172.21.15.129:0/3421753392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40109b10 0x7f7d40114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.297+0000 7f7d448f4640 1 --2- 172.21.15.129:0/3421753392 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d401091f0 0x7f7d401095d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.297+0000 7f7d448f4640 1 -- 172.21.15.129:0/3421753392 >> 172.21.15.129:0/3421753392 conn(0x7f7d4006e140 msgr2=0x7f7d4006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:47.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.299+0000 7f7d448f4640 1 -- 172.21.15.129:0/3421753392 shutdown_connections 2024-09-05T01:20:47.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.299+0000 7f7d448f4640 1 -- 172.21.15.129:0/3421753392 wait complete. 2024-09-05T01:20:47.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.300+0000 7f7d448f4640 1 Processor -- start 2024-09-05T01:20:47.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.300+0000 7f7d448f4640 1 -- start start 2024-09-05T01:20:47.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d401091f0 0x7f7d40114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d40109b10 0x7f7d4010b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4011a650 con 0x7f7d40114ab0 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4011a4d0 con 0x7f7d40109b10 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d448f4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7d4011a7d0 con 0x7f7d401091f0 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d3ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d3ed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55778/0 (socket says 172.21.15.129:55778) 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d3ed9d640 1 -- 172.21.15.129:0/1932795413 learned_addr learned my addr 172.21.15.129:0/1932795413 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.301+0000 7f7d3e59c640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d401091f0 0x7f7d40114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 -- 172.21.15.129:0/1932795413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d401091f0 msgr2=0x7f7d40114050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d401091f0 0x7f7d40114050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 -- 172.21.15.129:0/1932795413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d40109b10 msgr2=0x7f7d4010b010 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d40109b10 0x7f7d4010b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 -- 172.21.15.129:0/1932795413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4010bb10 con 0x7f7d40114ab0 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d3ed9d640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800cab0 tx=0x7f7d3800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38007e10 con 0x7f7d40114ab0 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.302+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4010be00 con 0x7f7d40114ab0 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.303+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d4010fdc0 con 0x7f7d40114ab0 2024-09-05T01:20:47.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.303+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d3800f900 con 0x7f7d40114ab0 2024-09-05T01:20:47.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.303+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d38018970 con 0x7f7d40114ab0 2024-09-05T01:20:47.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.304+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d20005290 con 0x7f7d40114ab0 2024-09-05T01:20:47.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.308+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7d3800f450 con 0x7f7d40114ab0 2024-09-05T01:20:47.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.309+0000 7f7d2f7fe640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 0x7f7d280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:47.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.309+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f7d380983d0 con 0x7f7d40114ab0 2024-09-05T01:20:47.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.309+0000 7f7d3e59c640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 0x7f7d280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:47.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.310+0000 7f7d3e59c640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 0x7f7d280a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006010 tx=0x7f7d30000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:47.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.315+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d38014030 con 0x7f7d40114ab0 2024-09-05T01:20:47.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:46 smithi183 ceph-mon[34589]: pgmap v515: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 1.10k op/s 2024-09-05T01:20:47.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.729+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d20005580 con 0x7f7d40114ab0 2024-09-05T01:20:47.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.731+0000 7f7d2f7fe640 1 -- 172.21.15.129:0/1932795413 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f7d38014030 con 0x7f7d40114ab0 2024-09-05T01:20:47.733 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:20:47.733 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:20:47.734 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 msgr2=0x7f7d280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 0x7f7d280a50c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006010 tx=0x7f7d30000bc0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 msgr2=0x7f7d4010b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800cab0 tx=0x7f7d3800cf70 comp rx=0 tx=0).stop 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 shutdown_connections 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7d280a2c00 0x7f7d280a50c0 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7d30006010 tx=0x7f7d30000bc0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7d40114ab0 0x7f7d4010b550 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.737 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7d40109b10 0x7f7d4010b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 --2- 172.21.15.129:0/1932795413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7d401091f0 0x7f7d40114050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:47.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.735+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 >> 172.21.15.129:0/1932795413 conn(0x7f7d4006e140 msgr2=0x7f7d40070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:47.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.736+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 shutdown_connections 2024-09-05T01:20:47.738 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:47.736+0000 7f7d448f4640 1 -- 172.21.15.129:0/1932795413 wait complete. 2024-09-05T01:20:48.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:48 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1932795413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:20:48.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1932795413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:20:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:48 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1932795413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:20:48.618 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0' 2024-09-05T01:20:48.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 154.3 --await-for 45.0 2024-09-05T01:20:49.081 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:20:49.391 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:49 smithi129 ceph-mon[30979]: pgmap v516: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-05T01:20:49.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:49 smithi183 ceph-mon[34589]: pgmap v516: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-05T01:20:49.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:49 smithi194 ceph-mon[34941]: pgmap v516: 129 pgs: 129 active+clean; 1023 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 11 MiB/s wr, 1.16k op/s 2024-09-05T01:20:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 225.0 s, 914.9 tps, lat 109.118 ms stddev 34.952 2024-09-05T01:20:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:50 smithi129 ceph-mon[30979]: pgmap v517: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-09-05T01:20:50.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:50 smithi183 ceph-mon[34589]: pgmap v517: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-09-05T01:20:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:50 smithi194 ceph-mon[34941]: pgmap v517: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.05k op/s 2024-09-05T01:20:52.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 -- 172.21.15.129:0/1286710819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb941036e0 msgr2=0x7fbb94107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/1286710819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb941036e0 0x7fbb94107b90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009930 tx=0x7fbb7c02a420 comp rx=0 tx=0).stop 2024-09-05T01:20:52.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 -- 172.21.15.129:0/1286710819 shutdown_connections 2024-09-05T01:20:52.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/1286710819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 0x7fbb9410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/1286710819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb941036e0 0x7fbb94107b90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/1286710819 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb94102d30 0x7fbb94103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 -- 172.21.15.129:0/1286710819 >> 172.21.15.129:0/1286710819 conn(0x7fbb940fc7d0 msgr2=0x7fbb940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 -- 172.21.15.129:0/1286710819 shutdown_connections 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 -- 172.21.15.129:0/1286710819 wait complete. 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.139+0000 7fbb99c7c640 1 Processor -- start 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 -- start start 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb941036e0 0x7fbb94198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 0x7fbb9419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111f60 con 0x7fbb94102d30 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb94111de0 con 0x7fbb941082a0 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb99c7c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941120e0 con 0x7fbb941036e0 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55802/0 (socket says 172.21.15.129:55802) 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 -- 172.21.15.129:0/299490001 learned_addr learned my addr 172.21.15.129:0/299490001 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb93fff640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 0x7fbb9419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb8affd640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb941036e0 0x7fbb94198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 -- 172.21.15.129:0/299490001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb941036e0 msgr2=0x7fbb94198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb941036e0 0x7fbb94198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 -- 172.21.15.129:0/299490001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 msgr2=0x7fbb9419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 0x7fbb9419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.140+0000 7fbb937fe640 1 -- 172.21.15.129:0/299490001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9419d4d0 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb937fe640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e9f0 tx=0x7fbb8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb917fa640 1 -- 172.21.15.129:0/299490001 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8400cd80 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb917fa640 1 -- 172.21.15.129:0/299490001 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb840054d0 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9419d7c0 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb917fa640 1 -- 172.21.15.129:0/299490001 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb84010490 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.141+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb941a50a0 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.144+0000 7fbb9c055740 1 Processor -- start 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.144+0000 7fbb917fa640 1 -- 172.21.15.129:0/299490001 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbb84005020 con 0x7fbb94102d30 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.144+0000 7fbb9c055740 1 -- start start 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.144+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ad2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55816/0 (socket says 172.21.15.129:55816) 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 -- 172.21.15.129:0/760304188 learned_addr learned my addr 172.21.15.129:0/760304188 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb9c055740 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104adfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560104a382a0 con 0x560104ad3080 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601049dfb10 con 0x560104add940 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560104a34260 con 0x560104ad2760 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb93fff640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ad2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb8affd640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104adfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb917fa640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 0x7fbb800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 -- 172.21.15.129:0/760304188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 msgr2=0x560104ad2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ad2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 -- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 msgr2=0x560104adfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104adfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 -- 172.21.15.129:0/760304188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601047a9b90 con 0x560104ad3080 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb8affd640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104adfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb93fff640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 0x7fbb800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb917fa640 1 -- 172.21.15.129:0/299490001 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbb840cf780 con 0x7fbb94102d30 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.145+0000 7fbb937fe640 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbb8406d340 tx=0x7fbb8406ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d1f84b54c3d631 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb89ffb640 1 -- 172.21.15.129:0/760304188 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8409b530 con 0x560104ad3080 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb89ffb640 1 -- 172.21.15.129:0/760304188 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8406d180 con 0x560104ad3080 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb89ffb640 1 -- 172.21.15.129:0/760304188 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb840a0ab0 con 0x560104ad3080 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 msgr2=0x560104add400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fbb8406d340 tx=0x7fbb8406ac10 comp rx=0 tx=0).stop 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb93fff640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 0x7fbb800a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005fd0 tx=0x7fbb8c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 shutdown_connections 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104adfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104add400 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 --2- 172.21.15.129:0/760304188 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ad2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 >> 172.21.15.129:0/760304188 conn(0x560104acecf0 msgr2=0x560104acf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 shutdown_connections 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.146+0000 7fbb9c055740 1 -- 172.21.15.129:0/760304188 wait complete. 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 -- start start 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 Processor -- start 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104ad2760 0x560104ada7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 0x560104adb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560104ae0510 con 0x560104ad3080 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5601049c06e0 con 0x560104ad2760 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560104ae0690 con 0x560104add940 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb9c055740 10 client.0 _ll_register_callbacks cb 0x5601049e4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55824/0 (socket says 172.21.15.129:55824) 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb8affd640 1 -- 172.21.15.129:0/395926827 learned_addr learned my addr 172.21.15.129:0/395926827 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb937fe640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104ad2760 0x560104ada7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.147+0000 7fbb93fff640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 0x560104adb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 -- 172.21.15.129:0/395926827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 msgr2=0x560104adb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 0x560104adb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 -- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104ad2760 msgr2=0x560104ada7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104ad2760 0x560104ada7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 -- 172.21.15.129:0/395926827 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560104ae9980 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb93fff640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 0x560104adb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb8affd640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbb940697d0 tx=0x7fbb7c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bb5aea15b15bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c035040 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbb7c005450 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560104a459d0 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbb7c005650 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 10 client.15912 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560104aec280 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560104a459d0 con 0x560104ad3080 2024-09-05T01:20:52.562 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbb7c003b20 con 0x560104ad3080 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 10 client.15912 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 10 client.15912 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.148+0000 7fbb9c055740 20 client.15912 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104aed790 0x560104aefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb9c055740 4 client.15912 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560104a3fd20 con 0x560104aed790 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb937fe640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104aed790 0x560104aefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb937fe640 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104aed790 0x560104aefb80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4194f5b32ecae89c server_cookie=1ddb7094ec0b2a59 in_seq=0 out_seq=0 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.149+0000 7fbb72ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.150+0000 7fbb72ffd640 1 -- 172.21.15.129:0/395926827 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560104a3fd20 con 0x560104aed790 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.150+0000 7fbb72ffd640 10 client.15912 handle_command_reply: tid=0 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 client.15912 shutdown 2024-09-05T01:20:52.563 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 msgr2=0x560104adacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbb940697d0 tx=0x7fbb7c0043b0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 shutdown_connections 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104aed790 0x560104aefb80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104add940 0x560104adb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad3080 0x560104adacf0 secure :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbb940697d0 tx=0x7fbb7c0043b0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 --2- 172.21.15.129:0/395926827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104ad2760 0x560104ada7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 >> 172.21.15.129:0/395926827 conn(0x560104acecf0 msgr2=0x560104ad4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.151+0000 7fbb72ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.152+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 shutdown_connections 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.153+0000 7fbb9c055740 1 -- 172.21.15.129:0/395926827 wait complete. 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.154+0000 7fbb9c055740 20 client.15912 trim_cache size 0 max 16384 2024-09-05T01:20:52.564 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.193+0000 7fbb9c055740 1 Processor -- start 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.194+0000 7fbb9c055740 1 -- start start 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.194+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad2760 0x560104adc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104ad9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb9c055740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941af580 con 0x560104ad2760 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104ad9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.565 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 18, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "ef602960": { 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 18, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:20:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 154.3, 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:20:52.567 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44980/0 (socket says 172.21.15.129:44980) 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.195+0000 7fbb93fff640 1 -- 172.21.15.129:0/316453723 learned_addr learned my addr 172.21.15.129:0/316453723 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.196+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ace50 con 0x560104add940 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.196+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941aaca0 con 0x560104adb080 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.196+0000 7fbb937fe640 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad2760 0x560104adc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 -- 172.21.15.129:0/316453723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 msgr2=0x560104ad9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104ad9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 -- 172.21.15.129:0/316453723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad2760 msgr2=0x560104adc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.568 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad2760 0x560104adc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 -- 172.21.15.129:0/316453723 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560104aca260 con 0x560104adb080 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.197+0000 7fbb93fff640 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0125c0 tx=0x7fbb8c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c462cc5d532603d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.198+0000 7fbb897fa640 1 -- 172.21.15.129:0/316453723 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c021410 con 0x560104adb080 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.198+0000 7fbb897fa640 1 -- 172.21.15.129:0/316453723 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8c021a60 con 0x560104adb080 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.198+0000 7fbb897fa640 1 -- 172.21.15.129:0/316453723 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c0109f0 con 0x560104adb080 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 msgr2=0x560104af5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0125c0 tx=0x7fbb8c016000 comp rx=0 tx=0).stop 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 shutdown_connections 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104adb080 0x560104af5440 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104add940 0x560104ad9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 --2- 172.21.15.129:0/316453723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104ad2760 0x560104adc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.569 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 >> 172.21.15.129:0/316453723 conn(0x560104acecf0 msgr2=0x560104acde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.199+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 shutdown_connections 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.200+0000 7fbb9c055740 1 -- 172.21.15.129:0/316453723 wait complete. 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.200+0000 7fbb9c055740 1 -- start start 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.200+0000 7fbb9c055740 1 Processor -- start 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104adb080 0x560104ae22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104add940 0x560104ae2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ab310 con 0x560104add940 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ac780 con 0x560104adb080 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbb941ab9e0 con 0x560104ad2760 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.201+0000 7fbb9c055740 10 client.0 _ll_register_callbacks cb 0x7fbb7c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.202+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:44996/0 (socket says 172.21.15.129:44996) 2024-09-05T01:20:52.570 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.202+0000 7fbb937fe640 1 -- 172.21.15.129:0/3375044308 learned_addr learned my addr 172.21.15.129:0/3375044308 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.202+0000 7fbb93fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104add940 0x560104ae2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.202+0000 7fbb8affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104adb080 0x560104ae22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 -- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104adb080 msgr2=0x560104ae22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104adb080 0x560104ae22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 -- 172.21.15.129:0/3375044308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104add940 msgr2=0x560104ae2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104add940 0x560104ae2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 -- 172.21.15.129:0/3375044308 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560104ae3060 con 0x560104ad2760 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb937fe640 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb8409b8d0 tx=0x7fbb84009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13dd9549a17219c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.203+0000 7fbb72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.206+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb840a1110 con 0x560104ad2760 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.206+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbb840afdb0 con 0x560104ad2760 2024-09-05T01:20:52.571 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.206+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb8c021a60 con 0x560104ad2760 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.207+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbb84005310 con 0x560104ad2760 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.207+0000 7fbb9c055740 10 client.25423 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.207+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560104b593f0 con 0x560104ad2760 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.207+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb8c021a60 con 0x560104ad2760 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbb840a1350 con 0x560104ad2760 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 10 client.25423 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 10 client.25423 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 20 client.25423 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104b59730 0x560104b5bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 4 client.25423 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}] 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.208+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.3, "await_for": 45.0}) -- 0x5601049bbec0 con 0x560104b59730 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.209+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104b59730 0x560104b5bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:52.572 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.210+0000 7fbb8affd640 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104b59730 0x560104b5bb20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9bdeceb2136965 server_cookie=2014cd356b061486 in_seq=0 out_seq=0 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.210+0000 7fbb72ffd640 10 client.25423 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb72ffd640 1 -- 172.21.15.129:0/3375044308 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601049bbec0 con 0x560104b59730 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb72ffd640 10 client.25423 handle_command_reply: tid=0 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 client.25423 shutdown 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 msgr2=0x560104ae5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbb8409b8d0 tx=0x7fbb84009310 comp rx=0 tx=0).stop 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 shutdown_connections 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560104b59730 0x560104b5bb20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560104add940 0x560104ae2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560104adb080 0x560104ae22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 --2- 172.21.15.129:0/3375044308 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560104ad2760 0x560104ae5fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb72ffd640 0 client.25423 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:52.573 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 >> 172.21.15.129:0/3375044308 conn(0x560104acecf0 msgr2=0x560104ada070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 shutdown_connections 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 1 -- 172.21.15.129:0/3375044308 wait complete. 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.490+0000 7fbb9c055740 20 client.25423 trim_cache size 0 max 16384 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 msgr2=0x7fbb800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 0x7fbb800a50c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c005fd0 tx=0x7fbb8c004a40 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 msgr2=0x7fbb94198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbb8400e9f0 tx=0x7fbb8400edf0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 shutdown_connections 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb800a2c00 0x7fbb800a50c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbb941082a0 0x7fbb9419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbb941036e0 0x7fbb94198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 --2- 172.21.15.129:0/299490001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbb94102d30 0x7fbb94198520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:52.574 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 >> 172.21.15.129:0/299490001 conn(0x7fbb940fc7d0 msgr2=0x7fbb94105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:52.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 shutdown_connections 2024-09-05T01:20:52.575 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:20:51.491+0000 7fbb99c7c640 1 -- 172.21.15.129:0/299490001 wait complete. 2024-09-05T01:20:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:52 smithi129 ceph-mon[30979]: pgmap v518: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-09-05T01:20:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:52 smithi194 ceph-mon[34941]: pgmap v518: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-09-05T01:20:53.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:52 smithi183 ceph-mon[34589]: pgmap v518: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.6 MiB/s wr, 960 op/s 2024-09-05T01:20:55.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 230.0 s, 161.1 tps, lat 128.209 ms stddev 60.322 2024-09-05T01:20:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:54 smithi129 ceph-mon[30979]: pgmap v519: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-05T01:20:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:54 smithi194 ceph-mon[34941]: pgmap v519: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-05T01:20:55.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:54 smithi183 ceph-mon[34589]: pgmap v519: 129 pgs: 129 active+clean; 1023 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 10 MiB/s wr, 1.03k op/s 2024-09-05T01:20:57.236 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:56 smithi129 ceph-mon[30979]: pgmap v520: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-09-05T01:20:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:56 smithi194 ceph-mon[34941]: pgmap v520: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-09-05T01:20:57.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:56 smithi183 ceph-mon[34589]: pgmap v520: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.0 MiB/s wr, 787 op/s 2024-09-05T01:20:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:20:58 smithi129 ceph-mon[30979]: pgmap v521: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 632 op/s 2024-09-05T01:20:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:20:58 smithi194 ceph-mon[34941]: pgmap v521: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 632 op/s 2024-09-05T01:20:59.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:20:58 smithi183 ceph-mon[34589]: pgmap v521: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 632 op/s 2024-09-05T01:20:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 -- 172.21.15.129:0/3682903274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c1025f0 msgr2=0x7faf9c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:59.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 --2- 172.21.15.129:0/3682903274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c1025f0 0x7faf9c108e80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faf94009ca0 tx=0x7faf9402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 -- 172.21.15.129:0/3682903274 shutdown_connections 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 --2- 172.21.15.129:0/3682903274 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c1025f0 0x7faf9c108e80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 --2- 172.21.15.129:0/3682903274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c101c30 0x7faf9c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 --2- 172.21.15.129:0/3682903274 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c100a40 0x7faf9c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 -- 172.21.15.129:0/3682903274 >> 172.21.15.129:0/3682903274 conn(0x7faf9c0fc1e0 msgr2=0x7faf9c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 -- 172.21.15.129:0/3682903274 shutdown_connections 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.975+0000 7fafa11c4640 1 -- 172.21.15.129:0/3682903274 wait complete. 2024-09-05T01:20:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 Processor -- start 2024-09-05T01:20:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 -- start start 2024-09-05T01:20:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c101c30 0x7faf9c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c1025f0 0x7faf9c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10c550 con 0x7faf9c100a40 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10c3d0 con 0x7faf9c1025f0 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.976+0000 7fafa11c4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c10c6d0 con 0x7faf9c101c30 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42642/0 (socket says 172.21.15.129:42642) 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 -- 172.21.15.129:0/1877389451 learned_addr learned my addr 172.21.15.129:0/1877389451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9b59e640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c1025f0 0x7faf9c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9a59c640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c101c30 0x7faf9c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 -- 172.21.15.129:0/1877389451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c101c30 msgr2=0x7faf9c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c101c30 0x7faf9c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 -- 172.21.15.129:0/1877389451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c1025f0 msgr2=0x7faf9c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c1025f0 0x7faf9c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 -- 172.21.15.129:0/1877389451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9c19f890 con 0x7faf9c100a40 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00d770 tx=0x7faf8c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf83fff640 1 -- 172.21.15.129:0/1877389451 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c017070 con 0x7faf9c100a40 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9c19fb20 con 0x7faf9c100a40 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf83fff640 1 -- 172.21.15.129:0/1877389451 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf8c009cd0 con 0x7faf9c100a40 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7faf83fff640 1 -- 172.21.15.129:0/1877389451 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c003c60 con 0x7faf9c100a40 2024-09-05T01:20:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.977+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf9c1a00c0 con 0x7faf9c100a40 2024-09-05T01:20:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.979+0000 7faf83fff640 1 -- 172.21.15.129:0/1877389451 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faf8c009820 con 0x7faf9c100a40 2024-09-05T01:20:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 Processor -- start 2024-09-05T01:20:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 -- start start 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16969a0 0x55adf1696da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf169e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55adf15fa2f0 con 0x55adf16972e0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55adf15a2270 con 0x55adf16969a0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7fafa4038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adf15f62b0 con 0x55adf169c370 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf83fff640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 0x7faf840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9b59e640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 0x7faf840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42654/0 (socket says 172.21.15.129:42654) 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9ad9d640 1 -- 172.21.15.129:0/750410106 learned_addr learned my addr 172.21.15.129:0/750410106 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9b59e640 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16969a0 0x55adf1696da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf9a59c640 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf169e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.980+0000 7faf83fff640 1 -- 172.21.15.129:0/1877389451 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf8c0cecf0 con 0x7faf9c100a40 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 -- 172.21.15.129:0/750410106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 msgr2=0x55adf169e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf169e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 -- 172.21.15.129:0/750410106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16969a0 msgr2=0x55adf1696da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16969a0 0x55adf1696da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 -- 172.21.15.129:0/750410106 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf15a1340 con 0x55adf16972e0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9b59e640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 0x7faf840a50f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf94009a60 tx=0x7faf94009890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0a3880 tx=0x7faf8c06e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5627f429d97f711d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf817fa640 1 -- 172.21.15.129:0/750410106 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c020070 con 0x55adf16972e0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf817fa640 1 -- 172.21.15.129:0/750410106 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf8c009b10 con 0x55adf16972e0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.981+0000 7faf817fa640 1 -- 172.21.15.129:0/750410106 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c099440 con 0x55adf16972e0 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- 172.21.15.129:0/750410106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 msgr2=0x55adf169bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0a3880 tx=0x7faf8c06e200 comp rx=0 tx=0).stop 2024-09-05T01:20:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- 172.21.15.129:0/750410106 shutdown_connections 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf169e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf169bda0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- 172.21.15.129:0/750410106 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16969a0 0x55adf1696da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- 172.21.15.129:0/750410106 >> 172.21.15.129:0/750410106 conn(0x55adf1692010 msgr2=0x55adf1692410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- 172.21.15.129:0/750410106 shutdown_connections 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- 172.21.15.129:0/750410106 wait complete. 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- start start 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 Processor -- start 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16972e0 0x55adf16a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf16a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55adf169f340 con 0x55adf16969a0 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55adf1583290 con 0x55adf16972e0 2024-09-05T01:20:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55adf169f4c0 con 0x55adf169c370 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.982+0000 7fafa4038740 10 client.0 _ll_register_callbacks cb 0x55adf15b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42660/0 (socket says 172.21.15.129:42660) 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 -- 172.21.15.129:0/2982655303 learned_addr learned my addr 172.21.15.129:0/2982655303 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9b59e640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf16a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9a59c640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16972e0 0x55adf16a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 -- 172.21.15.129:0/2982655303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 msgr2=0x55adf16a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf16a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 -- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16972e0 msgr2=0x55adf16a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16972e0 0x55adf16a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 -- 172.21.15.129:0/2982655303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf130d210 con 0x55adf16969a0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf8c097b10 tx=0x7faf8c097500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10994e493f9c0ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:20:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c020030 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf8c0043b0 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55adf1607a20 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf8c0b4050 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7fafa4038740 10 client.15927 fetch_fsmap learned FSMap version 30 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.983+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55adf13d9620 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55adf1607a20 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf8c0bb070 con 0x55adf16969a0 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 20 client.15927 populate_metadata read hostname 'smithi129' 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf16ac470 0x55adf16ae860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:20:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 4 client.15927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adf1601d70 con 0x55adf16ac470 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7faf9a59c640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf16ac470 0x55adf16ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7faf7eff5640 10 client.15927 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.984+0000 7faf9a59c640 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf16ac470 0x55adf16ae860 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3962589bea043e3 server_cookie=7559ddbae87289ac in_seq=0 out_seq=0 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7faf7eff5640 1 -- 172.21.15.129:0/2982655303 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adf1601d70 con 0x55adf16ac470 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7faf7eff5640 10 client.15927 handle_command_reply: tid=0 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 client.15927 shutdown 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 msgr2=0x55adf16a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faf8c097b10 tx=0x7faf8c097500 comp rx=0 tx=0).stop 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 shutdown_connections 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf16ac470 0x55adf16ae860 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf169c370 0x55adf16a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf16972e0 0x55adf16a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7faf7eff5640 0 client.15927 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:20:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 --2- 172.21.15.129:0/2982655303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a0bf0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:20:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 >> 172.21.15.129:0/2982655303 conn(0x55adf1692010 msgr2=0x55adf1693820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:20:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 shutdown_connections 2024-09-05T01:20:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 1 -- 172.21.15.129:0/2982655303 wait complete. 2024-09-05T01:20:59.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:20:59.985+0000 7fafa4038740 20 client.15927 trim_cache size 0 max 16384 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 Processor -- start 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 -- start start 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16972e0 0x55adf16a8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf16a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1ad090 con 0x55adf16969a0 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1aa960 con 0x55adf169c370 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7fafa4038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1a87b0 con 0x55adf16972e0 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42670/0 (socket says 172.21.15.129:42670) 2024-09-05T01:21:00.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 -- 172.21.15.129:0/153817621 learned_addr learned my addr 172.21.15.129:0/153817621 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9b59e640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf16a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9a59c640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16972e0 0x55adf16a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 -- 172.21.15.129:0/153817621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16972e0 msgr2=0x55adf16a8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16972e0 0x55adf16a8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 -- 172.21.15.129:0/153817621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 msgr2=0x55adf16a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf16a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 -- 172.21.15.129:0/153817621 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf169fbc0 con 0x55adf16969a0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9b59e640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf16a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.000+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0cc750 tx=0x7faf8c09ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28e9bcd36f64850 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7faf80ff9640 1 -- 172.21.15.129:0/153817621 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c020030 con 0x55adf16969a0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7faf80ff9640 1 -- 172.21.15.129:0/153817621 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf8c09b840 con 0x55adf16969a0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7faf80ff9640 1 -- 172.21.15.129:0/153817621 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8c09f1a0 con 0x55adf16969a0 2024-09-05T01:21:00.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 -- 172.21.15.129:0/153817621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 msgr2=0x55adf16a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0cc750 tx=0x7faf8c09ad40 comp rx=0 tx=0).stop 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 -- 172.21.15.129:0/153817621 shutdown_connections 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf16a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16972e0 0x55adf16a8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 --2- 172.21.15.129:0/153817621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16969a0 0x55adf16a09a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 -- 172.21.15.129:0/153817621 >> 172.21.15.129:0/153817621 conn(0x55adf1692010 msgr2=0x55adf169bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 -- 172.21.15.129:0/153817621 shutdown_connections 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.001+0000 7fafa4038740 1 -- 172.21.15.129:0/153817621 wait complete. 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 -- start start 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 Processor -- start 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16969a0 0x55adf16a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf1715080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1a8e20 con 0x55adf16972e0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1aa290 con 0x55adf169c370 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf9c1a94f0 con 0x55adf16969a0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7fafa4038740 10 client.0 _ll_register_callbacks cb 0x7faf940316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42672/0 (socket says 172.21.15.129:42672) 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 -- 172.21.15.129:0/685848709 learned_addr learned my addr 172.21.15.129:0/685848709 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9b59e640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf1715080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 -- 172.21.15.129:0/685848709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16969a0 msgr2=0x55adf16a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16969a0 0x55adf16a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16969a0 0x55adf16a4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 -- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 msgr2=0x55adf1715080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf1715080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 -- 172.21.15.129:0/685848709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf16a82e0 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9b59e640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf1715080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf9a59c640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf90010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fc6eb29f9f1443 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.002+0000 7faf7eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9002b060 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf8c09b840 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf900049a0 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf90032050 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 10 client.15933 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55adf17165f0 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf8c09b840 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf90005130 con 0x55adf16972e0 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 10 client.15933 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:00.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 20 client.15933 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf1716f80 0x55adf17193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 4 client.15933 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55adf157e900 con 0x55adf1716f80 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.003+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf1716f80 0x55adf17193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.004+0000 7faf7eff5640 10 client.15933 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:00.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.004+0000 7faf9ad9d640 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf1716f80 0x55adf17193a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b35075a0052c86 server_cookie=854a92deb0b8a415 in_seq=0 out_seq=0 2024-09-05T01:21:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7faf7eff5640 1 -- 172.21.15.129:0/685848709 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55adf157e900 con 0x55adf1716f80 2024-09-05T01:21:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7faf7eff5640 10 client.15933 handle_command_reply: tid=0 2024-09-05T01:21:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 client.15933 shutdown 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 msgr2=0x55adf16a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7faf90000c00 tx=0x7faf90010af0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 shutdown_connections 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55adf1716f80 0x55adf17193a0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55adf169c370 0x55adf1715080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55adf16972e0 0x55adf16a51f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7faf7eff5640 0 client.15933 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 --2- 172.21.15.129:0/685848709 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55adf16969a0 0x55adf16a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 >> 172.21.15.129:0/685848709 conn(0x55adf1692010 msgr2=0x55adf16a09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 shutdown_connections 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.018+0000 7fafa4038740 1 -- 172.21.15.129:0/685848709 wait complete. 2024-09-05T01:21:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa4038740 20 client.15933 trim_cache size 0 max 16384 2024-09-05T01:21:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:21:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:21:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:21:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 msgr2=0x7faf840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 0x7faf840a50f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf94009a60 tx=0x7faf94009890 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 msgr2=0x7faf9c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00d770 tx=0x7faf8c00db70 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 shutdown_connections 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf840a2c30 0x7faf840a50f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf9c1025f0 0x7faf9c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf9c101c30 0x7faf9c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 --2- 172.21.15.129:0/1877389451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf9c100a40 0x7faf9c06a410 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 >> 172.21.15.129:0/1877389451 conn(0x7faf9c0fc1e0 msgr2=0x7faf9c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 shutdown_connections 2024-09-05T01:21:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:00.019+0000 7fafa11c4640 1 -- 172.21.15.129:0/1877389451 wait complete. 2024-09-05T01:21:00.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 235.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:21:01.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:00 smithi183 ceph-mon[34589]: pgmap v522: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-05T01:21:01.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:00 smithi129 ceph-mon[30979]: pgmap v522: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-05T01:21:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:00 smithi194 ceph-mon[34941]: pgmap v522: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-09-05T01:21:02.016 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ef602960', quiesce took 15.8 seconds. Will release after: 18.499999999999996 2024-09-05T01:21:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:02 smithi129 ceph-mon[30979]: pgmap v523: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T01:21:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:02 smithi194 ceph-mon[34941]: pgmap v523: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T01:21:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:02 smithi183 ceph-mon[34589]: pgmap v523: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T01:21:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 240.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:21:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:04 smithi129 ceph-mon[30979]: pgmap v524: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-05T01:21:05.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:04 smithi194 ceph-mon[34941]: pgmap v524: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-05T01:21:05.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:04 smithi183 ceph-mon[34589]: pgmap v524: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 MiB/s wr, 210 op/s 2024-09-05T01:21:07.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:06 smithi129 ceph-mon[30979]: pgmap v525: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 673 KiB/s wr, 79 op/s 2024-09-05T01:21:07.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:06 smithi194 ceph-mon[34941]: pgmap v525: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 673 KiB/s wr, 79 op/s 2024-09-05T01:21:07.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:06 smithi183 ceph-mon[34589]: pgmap v525: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 673 KiB/s wr, 79 op/s 2024-09-05T01:21:09.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:08 smithi129 ceph-mon[30979]: pgmap v526: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 162 KiB/s wr, 29 op/s 2024-09-05T01:21:09.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:08 smithi194 ceph-mon[34941]: pgmap v526: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 162 KiB/s wr, 29 op/s 2024-09-05T01:21:09.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:08 smithi183 ceph-mon[34589]: pgmap v526: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 162 KiB/s wr, 29 op/s 2024-09-05T01:21:10.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 245.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:21:11.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:10 smithi129 ceph-mon[30979]: pgmap v527: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 KiB/s wr, 32 op/s 2024-09-05T01:21:11.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:10 smithi194 ceph-mon[34941]: pgmap v527: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 KiB/s wr, 32 op/s 2024-09-05T01:21:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:10 smithi183 ceph-mon[34589]: pgmap v527: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 168 KiB/s wr, 32 op/s 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 -- 172.21.15.129:0/855876400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9541016d0 msgr2=0x7fc954108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 --2- 172.21.15.129:0/855876400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9541016d0 0x7fc954108f70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009ca0 tx=0x7fc94c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 -- 172.21.15.129:0/855876400 shutdown_connections 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 --2- 172.21.15.129:0/855876400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc9541016d0 0x7fc954108f70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 --2- 172.21.15.129:0/855876400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc954100d10 0x7fc954101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 --2- 172.21.15.129:0/855876400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954102070 0x7fc954102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 -- 172.21.15.129:0/855876400 >> 172.21.15.129:0/855876400 conn(0x7fc9540fb970 msgr2=0x7fc9540fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 -- 172.21.15.129:0/855876400 shutdown_connections 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.107+0000 7fc95bb49640 1 -- 172.21.15.129:0/855876400 wait complete. 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 Processor -- start 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 -- start start 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954100d10 0x7fc954196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9541016d0 0x7fc954197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c5e0 con 0x7fc954102070 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c460 con 0x7fc9541016d0 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95bb49640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc95410c760 con 0x7fc954100d10 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40752/0 (socket says 172.21.15.129:40752) 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 -- 172.21.15.129:0/998369548 learned_addr learned my addr 172.21.15.129:0/998369548 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc9590e4640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9541016d0 0x7fc954197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc9598e5640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954100d10 0x7fc954196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 -- 172.21.15.129:0/998369548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954100d10 msgr2=0x7fc954196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954100d10 0x7fc954196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 -- 172.21.15.129:0/998369548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9541016d0 msgr2=0x7fc954197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9541016d0 0x7fc954197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.108+0000 7fc95a0e6640 1 -- 172.21.15.129:0/998369548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95419e840 con 0x7fc954102070 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc95a0e6640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc94c031040 tx=0x7fc94c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc94affd640 1 -- 172.21.15.129:0/998369548 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c003c70 con 0x7fc954102070 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc94affd640 1 -- 172.21.15.129:0/998369548 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc94c003e10 con 0x7fc954102070 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc94affd640 1 -- 172.21.15.129:0/998369548 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c030c40 con 0x7fc954102070 2024-09-05T01:21:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95419ead0 con 0x7fc954102070 2024-09-05T01:21:13.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.109+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95419f070 con 0x7fc954102070 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc94affd640 1 -- 172.21.15.129:0/998369548 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fc94c004040 con 0x7fc954102070 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 Processor -- start 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 -- start start 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c04da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a93b682f0 con 0x559a93c052e0 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a93b10270 con 0x559a93c0a370 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a93b642b0 con 0x559a93c049a0 2024-09-05T01:21:13.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.112+0000 7fc9598e5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40758/0 (socket says 172.21.15.129:40758) 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 -- 172.21.15.129:0/1130348959 learned_addr learned my addr 172.21.15.129:0/1130348959 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c04da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 -- 172.21.15.129:0/1130348959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 msgr2=0x559a93c04da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c04da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc94affd640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 0x7fc9400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 -- 172.21.15.129:0/1130348959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 msgr2=0x559a93c0c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c0c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 -- 172.21.15.129:0/1130348959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a93b0f340 con 0x559a93c052e0 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc9598e5640 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc95000bc50 tx=0x7fc95000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bceb1488e7cb6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc95a0e6640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 0x7fc9400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc94affd640 1 -- 172.21.15.129:0/998369548 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fc94c0f0870 con 0x7fc954102070 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc93ffff640 1 -- 172.21.15.129:0/1130348959 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950015070 con 0x559a93c052e0 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc93ffff640 1 -- 172.21.15.129:0/1130348959 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9500054d0 con 0x559a93c052e0 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.113+0000 7fc93ffff640 1 -- 172.21.15.129:0/1130348959 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950011440 con 0x559a93c052e0 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1130348959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 msgr2=0x559a93c09da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95a0e6640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 0x7fc9400a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0b9090 tx=0x7fc94c0b2110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc95000bc50 tx=0x7fc95000f730 comp rx=0 tx=0).stop 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1130348959 shutdown_connections 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c09da0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1130348959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c04da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1130348959 >> 172.21.15.129:0/1130348959 conn(0x559a93c00010 msgr2=0x559a93c00410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1130348959 shutdown_connections 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1130348959 wait complete. 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 -- start start 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.114+0000 7fc95e9ba740 1 Processor -- start 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c0f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c0a370 0x559a93c16170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a93c0d340 con 0x559a93c049a0 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a93af1290 con 0x559a93c052e0 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a93c0d4c0 con 0x559a93c0a370 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc95e9ba740 10 client.0 _ll_register_callbacks cb 0x559a93b22140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40770/0 (socket says 172.21.15.129:40770) 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 -- 172.21.15.129:0/3344850757 learned_addr learned my addr 172.21.15.129:0/3344850757 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c0a370 0x559a93c16170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9598e5640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c0f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 -- 172.21.15.129:0/3344850757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c0a370 msgr2=0x559a93c16170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c0a370 0x559a93c16170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 -- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 msgr2=0x559a93c0f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c0f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 -- 172.21.15.129:0/3344850757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9387b210 con 0x559a93c049a0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc9590e4640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc944009f70 tx=0x7fc944005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b1cfd2ccc2db2d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc93d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.115+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944004250 con 0x559a93c049a0 2024-09-05T01:21:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9500054d0 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fc9440043f0 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc94401ee30 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 10 client.15945 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a93947620 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9500054d0 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc94401c570 con 0x559a93c049a0 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 10 client.15945 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 20 client.15945 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c1a880 0x559a93c1ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 4 client.15945 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a93b75a20 con 0x559a93c1a880 2024-09-05T01:21:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.116+0000 7fc9598e5640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c1a880 0x559a93c1ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc9598e5640 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c1a880 0x559a93c1ccf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a5f8e6185862ea server_cookie=bb2b8c52e367d62c in_seq=0 out_seq=0 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc93d7fa640 10 client.15945 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc93d7fa640 1 -- 172.21.15.129:0/3344850757 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a93b75a20 con 0x559a93c1a880 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc93d7fa640 10 client.15945 handle_command_reply: tid=0 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc95e9ba740 1 client.15945 shutdown 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 msgr2=0x559a93c0ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.117+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc944009f70 tx=0x7fc944005f90 comp rx=0 tx=0).stop 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 shutdown_connections 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c1a880 0x559a93c1ccf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c0a370 0x559a93c16170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c0f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/3344850757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c049a0 0x559a93c0ebf0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc93d7fa640 0 client.15945 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 >> 172.21.15.129:0/3344850757 conn(0x559a93c00010 msgr2=0x559a93c01820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 shutdown_connections 2024-09-05T01:21:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 1 -- 172.21.15.129:0/3344850757 wait complete. 2024-09-05T01:21:13.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.118+0000 7fc95e9ba740 20 client.15945 trim_cache size 0 max 16384 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.132+0000 7fc95e9ba740 1 Processor -- start 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 -- start start 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c0f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541aae90 con 0x559a93c052e0 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541a8760 con 0x559a93c0a370 2024-09-05T01:21:13.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541a65b0 con 0x559a93c049a0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40782/0 (socket says 172.21.15.129:40782) 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 -- 172.21.15.129:0/2918025008 learned_addr learned my addr 172.21.15.129:0/2918025008 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9590e4640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c0f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9590e4640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 -- 172.21.15.129:0/2918025008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 msgr2=0x559a93c0f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c0f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 -- 172.21.15.129:0/2918025008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 msgr2=0x559a93c215f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c215f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 -- 172.21.15.129:0/2918025008 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a93c0e730 con 0x559a93c052e0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9590e4640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc9598e5640 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc95000fc30 tx=0x7fc9500056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a88f3ae4049ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc93f7fe640 1 -- 172.21.15.129:0/2918025008 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950015040 con 0x559a93c052e0 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.133+0000 7fc93f7fe640 1 -- 172.21.15.129:0/2918025008 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc950004bf0 con 0x559a93c052e0 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc93f7fe640 1 -- 172.21.15.129:0/2918025008 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950025680 con 0x559a93c052e0 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- 172.21.15.129:0/2918025008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 msgr2=0x559a93c16130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fc95000fc30 tx=0x7fc9500056a0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- 172.21.15.129:0/2918025008 shutdown_connections 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c0a370 0x559a93c215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c052e0 0x559a93c16130 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/2918025008 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c0f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- 172.21.15.129:0/2918025008 >> 172.21.15.129:0/2918025008 conn(0x559a93c00010 msgr2=0x559a93bff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- 172.21.15.129:0/2918025008 shutdown_connections 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- 172.21.15.129:0/2918025008 wait complete. 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.134+0000 7fc95e9ba740 1 -- start start 2024-09-05T01:21:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 Processor -- start 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c13df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c14330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541a6c20 con 0x559a93c0a370 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541a8090 con 0x559a93c052e0 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541a72f0 con 0x559a93c049a0 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc95e9ba740 10 client.0 _ll_register_callbacks cb 0x7fc94401e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40798/0 (socket says 172.21.15.129:40798) 2024-09-05T01:21:13.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 -- 172.21.15.129:0/1864249556 learned_addr learned my addr 172.21.15.129:0/1864249556 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 -- 172.21.15.129:0/1864249556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 msgr2=0x559a93c13df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c13df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 -- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 msgr2=0x559a93c14330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c14330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 -- 172.21.15.129:0/1864249556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a93c15900 con 0x559a93c0a370 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc9590e4640 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc94401eac0 tx=0x7fc94401e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c7cd744be56d19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.135+0000 7fc93d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944035030 con 0x559a93c0a370 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fc94401cb80 con 0x559a93c0a370 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc950004bf0 con 0x559a93c0a370 2024-09-05T01:21:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc94403f050 con 0x559a93c0a370 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 10 client.15951 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a93c85b20 con 0x559a93c0a370 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc950004bf0 con 0x559a93c0a370 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fc944003c30 con 0x559a93c0a370 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 10 client.15951 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 20 client.15951 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c864d0 0x559a93c888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 4 client.15951 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:21:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a93b6fd70 con 0x559a93c864d0 2024-09-05T01:21:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.136+0000 7fc9598e5640 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c864d0 0x559a93c888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.137+0000 7fc93d7fa640 10 client.15951 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:13.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.137+0000 7fc9598e5640 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c864d0 0x559a93c888f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3909709a518eebf server_cookie=e9d2a7fc62de8678 in_seq=0 out_seq=0 2024-09-05T01:21:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc93d7fa640 1 -- 172.21.15.129:0/1864249556 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a93b6fd70 con 0x559a93c864d0 2024-09-05T01:21:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc93d7fa640 10 client.15951 handle_command_reply: tid=0 2024-09-05T01:21:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 client.15951 shutdown 2024-09-05T01:21:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 msgr2=0x559a93c152f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc94401eac0 tx=0x7fc94401e880 comp rx=0 tx=0).stop 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 shutdown_connections 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a93c864d0 0x559a93c888f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a93c0a370 0x559a93c152f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a93c052e0 0x559a93c14330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 --2- 172.21.15.129:0/1864249556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a93c049a0 0x559a93c13df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 >> 172.21.15.129:0/1864249556 conn(0x559a93c00010 msgr2=0x559a93c00e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc93d7fa640 0 client.15951 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 shutdown_connections 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 1 -- 172.21.15.129:0/1864249556 wait complete. 2024-09-05T01:21:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.150+0000 7fc95e9ba740 20 client.15951 trim_cache size 0 max 16384 2024-09-05T01:21:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:21:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:21:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:21:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 msgr2=0x7fc9400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 0x7fc9400a5170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0b9090 tx=0x7fc94c0b2110 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 msgr2=0x7fc95419e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc94c031040 tx=0x7fc94c009360 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 shutdown_connections 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fc9400a2cb0 0x7fc9400a5170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc954102070 0x7fc95419e1a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9541016d0 0x7fc954197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 --2- 172.21.15.129:0/998369548 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fc954100d10 0x7fc954196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 >> 172.21.15.129:0/998369548 conn(0x7fc9540fb970 msgr2=0x7fc9541050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 shutdown_connections 2024-09-05T01:21:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:13.151+0000 7fc95bb49640 1 -- 172.21.15.129:0/998369548 wait complete. 2024-09-05T01:21:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:12 smithi129 ceph-mon[30979]: pgmap v528: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 165 KiB/s wr, 30 op/s 2024-09-05T01:21:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:21:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:21:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:21:13.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:12 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:21:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:12 smithi194 ceph-mon[34941]: pgmap v528: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 165 KiB/s wr, 30 op/s 2024-09-05T01:21:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:21:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:21:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:21:13.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:12 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:21:13.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:12 smithi183 ceph-mon[34589]: pgmap v528: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 165 KiB/s wr, 30 op/s 2024-09-05T01:21:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:21:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:21:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:21:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:12 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:21:15.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 250.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:21:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:14 smithi129 ceph-mon[30979]: pgmap v529: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-05T01:21:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:14 smithi194 ceph-mon[34941]: pgmap v529: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-05T01:21:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:14 smithi183 ceph-mon[34589]: pgmap v529: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-05T01:21:17.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:16 smithi129 ceph-mon[30979]: pgmap v530: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 186 KiB/s wr, 31 op/s 2024-09-05T01:21:17.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:16 smithi194 ceph-mon[34941]: pgmap v530: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 186 KiB/s wr, 31 op/s 2024-09-05T01:21:17.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:16 smithi183 ceph-mon[34589]: pgmap v530: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 186 KiB/s wr, 31 op/s 2024-09-05T01:21:19.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:18 smithi129 ceph-mon[30979]: pgmap v531: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 200 KiB/s wr, 34 op/s 2024-09-05T01:21:19.254 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:18 smithi194 ceph-mon[34941]: pgmap v531: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 200 KiB/s wr, 34 op/s 2024-09-05T01:21:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:18 smithi183 ceph-mon[34589]: pgmap v531: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 200 KiB/s wr, 34 op/s 2024-09-05T01:21:20.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 255.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:21:20.518 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:21:20.770 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:21:21.160 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:20 smithi129 ceph-mon[30979]: pgmap v532: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 201 KiB/s wr, 35 op/s 2024-09-05T01:21:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:20 smithi194 ceph-mon[34941]: pgmap v532: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 201 KiB/s wr, 35 op/s 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 -- 172.21.15.129:0/746420966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0103760 msgr2=0x7f51a0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 --2- 172.21.15.129:0/746420966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0103760 0x7f51a0107ba0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f519c009930 tx=0x7f519c02a420 comp rx=0 tx=0).stop 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 -- 172.21.15.129:0/746420966 shutdown_connections 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 --2- 172.21.15.129:0/746420966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a01082e0 0x7f51a010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 --2- 172.21.15.129:0/746420966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0103760 0x7f51a0107ba0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 --2- 172.21.15.129:0/746420966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a0102db0 0x7f51a0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 -- 172.21.15.129:0/746420966 >> 172.21.15.129:0/746420966 conn(0x7f51a00fc7d0 msgr2=0x7f51a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 -- 172.21.15.129:0/746420966 shutdown_connections 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.243+0000 7f51a7901640 1 -- 172.21.15.129:0/746420966 wait complete. 2024-09-05T01:21:21.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.244+0000 7f51a7901640 1 Processor -- start 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.244+0000 7f51a7901640 1 -- start start 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.244+0000 7f51a7901640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.244+0000 7f51a7901640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a0103760 0x7f51a0100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.244+0000 7f51a7901640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a01082e0 0x7f51a0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a7901640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0111f80 con 0x7f51a0102db0 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a7901640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0111e00 con 0x7f51a0103760 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a7901640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f51a0112100 con 0x7f51a01082e0 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40808/0 (socket says 172.21.15.129:40808) 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 -- 172.21.15.129:0/2139639539 learned_addr learned my addr 172.21.15.129:0/2139639539 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a4e9c640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a0103760 0x7f51a0100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:21.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a5e9e640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a01082e0 0x7f51a0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 -- 172.21.15.129:0/2139639539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a01082e0 msgr2=0x7f51a0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a01082e0 0x7f51a0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 -- 172.21.15.129:0/2139639539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a0103760 msgr2=0x7f51a0100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a0103760 0x7f51a0100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 -- 172.21.15.129:0/2139639539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a01a6a50 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a569d640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f519000b4a0 tx=0x7f519000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190014070 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5190010ac0 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a01a6d40 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190017c30 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.245+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a01a7240 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.246+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a006a7d0 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.248+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5190017070 con 0x7f51a0102db0 2024-09-05T01:21:21.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.248+0000 7f51967fc640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 0x7f518c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:21.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.248+0000 7f51a4e9c640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 0x7f518c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:21.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.248+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f51900ceb90 con 0x7f51a0102db0 2024-09-05T01:21:21.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.249+0000 7f51a4e9c640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 0x7f518c0a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f519c009900 tx=0x7f519c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:21.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.256+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f519009b6e0 con 0x7f51a0102db0 2024-09-05T01:21:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:20 smithi183 ceph-mon[34589]: pgmap v532: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 201 KiB/s wr, 35 op/s 2024-09-05T01:21:21.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.473+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a0103190 con 0x7f51a0102db0 2024-09-05T01:21:21.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.473+0000 7f51967fc640 1 -- 172.21.15.129:0/2139639539 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f51a0103190 con 0x7f51a0102db0 2024-09-05T01:21:21.474 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:21:21.475 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:21:21.475 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:21:21.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 msgr2=0x7f518c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 0x7f518c0a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f519c009900 tx=0x7f519c000bc0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 msgr2=0x7f51a0100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f519000b4a0 tx=0x7f519000b960 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 shutdown_connections 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f518c0a2c00 0x7f518c0a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f51a01082e0 0x7f51a0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f51a0103760 0x7f51a0100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 --2- 172.21.15.129:0/2139639539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f51a0102db0 0x7f51a0100a10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:21.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 >> 172.21.15.129:0/2139639539 conn(0x7f51a00fc7d0 msgr2=0x7f51a0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:21.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 shutdown_connections 2024-09-05T01:21:21.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:21.476+0000 7f51a7901640 1 -- 172.21.15.129:0/2139639539 wait complete. 2024-09-05T01:21:22.192 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:21 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2139639539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:21:22.227 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id ef602960 --release --await' 2024-09-05T01:21:22.227 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id ef602960 --release --await 2024-09-05T01:21:22.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:21 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2139639539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:21:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:21 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2139639539' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:21:22.488 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 -- 172.21.15.129:0/85897469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44100a90 msgr2=0x7fed44100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 --2- 172.21.15.129:0/85897469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44100a90 0x7fed44100ef0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed3402a420 comp rx=0 tx=0).stop 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 -- 172.21.15.129:0/85897469 shutdown_connections 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 --2- 172.21.15.129:0/85897469 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44101430 0x7fed4410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 --2- 172.21.15.129:0/85897469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44100a90 0x7fed44100ef0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 --2- 172.21.15.129:0/85897469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44106a90 0x7fed44106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 -- 172.21.15.129:0/85897469 >> 172.21.15.129:0/85897469 conn(0x7fed440fc7d0 msgr2=0x7fed440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.014 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 -- 172.21.15.129:0/85897469 shutdown_connections 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.011+0000 7fed4b60d640 1 -- 172.21.15.129:0/85897469 wait complete. 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 Processor -- start 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 -- start start 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 0x7fed44198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44101430 0x7fed44198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed44110d60 con 0x7fed44106a90 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed44110be0 con 0x7fed44101430 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed4b60d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed44110ee0 con 0x7fed44100a90 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed49baa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed49baa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37918/0 (socket says 172.21.15.129:37918) 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed49baa640 1 -- 172.21.15.129:0/1561349919 learned_addr learned my addr 172.21.15.129:0/1561349919 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:23.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed48ba8640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44101430 0x7fed44198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.012+0000 7fed493a9640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 0x7fed44198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 -- 172.21.15.129:0/1561349919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 msgr2=0x7fed44198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 0x7fed44198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 -- 172.21.15.129:0/1561349919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44101430 msgr2=0x7fed44198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44101430 0x7fed44198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 -- 172.21.15.129:0/1561349919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4419d3e0 con 0x7fed44106a90 2024-09-05T01:21:23.016 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:22 smithi129 ceph-mon[30979]: pgmap v533: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:21:23.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed493a9640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 0x7fed44198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:21:23.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.013+0000 7fed49baa640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00b970 tx=0x7fed3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.014+0000 7fed3a7fc640 1 -- 172.21.15.129:0/1561349919 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c019070 con 0x7fed44106a90 2024-09-05T01:21:23.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.014+0000 7fed3a7fc640 1 -- 172.21.15.129:0/1561349919 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c014440 con 0x7fed44106a90 2024-09-05T01:21:23.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.014+0000 7fed3a7fc640 1 -- 172.21.15.129:0/1561349919 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0075d0 con 0x7fed44106a90 2024-09-05T01:21:23.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.014+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4419d6d0 con 0x7fed44106a90 2024-09-05T01:21:23.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.014+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a4fb0 con 0x7fed44106a90 2024-09-05T01:21:23.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.016+0000 7fed3a7fc640 1 -- 172.21.15.129:0/1561349919 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fed3c01c070 con 0x7fed44106a90 2024-09-05T01:21:23.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.017+0000 7fed3a7fc640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 0x7fed300a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.017+0000 7fed3a7fc640 1 -- 172.21.15.129:0/1561349919 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed3c0ce910 con 0x7fed44106a90 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.017+0000 7fed493a9640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 0x7fed300a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.017+0000 7fed493a9640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 0x7fed300a5190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fed40009c20 tx=0x7fed40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 Processor -- start 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 -- start start 2024-09-05T01:21:23.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f19630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641d6e71620 con 0x5641d6f0c980 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641d6e18e50 con 0x5641d6f17240 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5641d6e6d5e0 con 0x5641d6f0c060 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed48ba8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed48ba8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37928/0 (socket says 172.21.15.129:37928) 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.018+0000 7fed48ba8640 1 -- 172.21.15.129:0/3924251802 learned_addr learned my addr 172.21.15.129:0/3924251802 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed493a9640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed49baa640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f19630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 -- 172.21.15.129:0/3924251802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 msgr2=0x5641d6f0c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f0c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 -- 172.21.15.129:0/3924251802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 msgr2=0x5641d6f19630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f19630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 -- 172.21.15.129:0/3924251802 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d6e09d40 con 0x5641d6f0c980 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed49baa640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f19630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed48ba8640 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a21f6cfcf1ed91e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed2f7fe640 1 -- 172.21.15.129:0/3924251802 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34040070 con 0x5641d6f0c980 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed2f7fe640 1 -- 172.21.15.129:0/3924251802 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed340316a0 con 0x5641d6f0c980 2024-09-05T01:21:23.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.019+0000 7fed2f7fe640 1 -- 172.21.15.129:0/3924251802 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035040 con 0x5641d6f0c980 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- 172.21.15.129:0/3924251802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 msgr2=0x5641d6f16d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34005fd0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- 172.21.15.129:0/3924251802 shutdown_connections 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f19630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f16d00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/3924251802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f0c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- 172.21.15.129:0/3924251802 >> 172.21.15.129:0/3924251802 conn(0x5641d6f085f0 msgr2=0x5641d6f089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- 172.21.15.129:0/3924251802 shutdown_connections 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- 172.21.15.129:0/3924251802 wait complete. 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 -- start start 2024-09-05T01:21:23.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.020+0000 7fed4d9e6740 1 Processor -- start 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f14610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 0x5641d6f14b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641d6f19e10 con 0x5641d6f0c060 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5641d6df9f10 con 0x5641d6f0c980 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5641d6f19f90 con 0x5641d6f17240 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed4d9e6740 10 client.0 _ll_register_callbacks cb 0x5641d6e1ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37932/0 (socket says 172.21.15.129:37932) 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 -- 172.21.15.129:0/1532098794 learned_addr learned my addr 172.21.15.129:0/1532098794 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed48ba8640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f14610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed49baa640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 0x5641d6f14b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 -- 172.21.15.129:0/1532098794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 msgr2=0x5641d6f14b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 0x5641d6f14b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 -- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 msgr2=0x5641d6f14610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f14610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 -- 172.21.15.129:0/1532098794 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d6b80580 con 0x5641d6f0c060 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed49baa640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 0x5641d6f14b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed493a9640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fed40003870 tx=0x7fed40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7acff469eaea13 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.021+0000 7fed2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40025070 con 0x5641d6f0c060 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed400129d0 con 0x5641d6f0c060 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed340316a0 con 0x5641d6f0c060 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed40031050 con 0x5641d6f0c060 2024-09-05T01:21:23.028 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 10 client.15969 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641d6c4a960 con 0x5641d6f0c060 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed340316a0 con 0x5641d6f0c060 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fed400293b0 con 0x5641d6f0c060 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 10 client.15969 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 10 client.15969 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 20 client.15969 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f26cb0 0x5641d6f29110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 4 client.15969 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641d6e7eb90 con 0x5641d6f26cb0 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.022+0000 7fed48ba8640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f26cb0 0x5641d6f29110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.023+0000 7fed2cff9640 10 client.15969 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.023+0000 7fed48ba8640 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f26cb0 0x5641d6f29110 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e065535636f2d12b server_cookie=68d46a02ffa929f in_seq=0 out_seq=0 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.023+0000 7fed2cff9640 1 -- 172.21.15.129:0/1532098794 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641d6e7eb90 con 0x5641d6f26cb0 2024-09-05T01:21:23.029 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.023+0000 7fed2cff9640 10 client.15969 handle_command_reply: tid=0 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.023+0000 7fed4d9e6740 1 client.15969 shutdown 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 msgr2=0x5641d6f140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fed40003870 tx=0x7fed40004730 comp rx=0 tx=0).stop 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 shutdown_connections 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f26cb0 0x5641d6f29110 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f17240 0x5641d6f14b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f14610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/1532098794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c060 0x5641d6f140d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 >> 172.21.15.129:0/1532098794 conn(0x5641d6f085f0 msgr2=0x5641d6f0e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed2cff9640 0 client.15969 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 shutdown_connections 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 1 -- 172.21.15.129:0/1532098794 wait complete. 2024-09-05T01:21:23.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.024+0000 7fed4d9e6740 20 client.15969 trim_cache size 0 max 16384 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.041+0000 7fed4d9e6740 1 Processor -- start 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.041+0000 7fed4d9e6740 1 -- start start 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f2e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b1700 con 0x5641d6f17240 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aefd0 con 0x5641d6f0c980 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ace20 con 0x5641d6f0c060 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37946/0 (socket says 172.21.15.129:37946) 2024-09-05T01:21:23.047 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 -- 172.21.15.129:0/160081899 learned_addr learned my addr 172.21.15.129:0/160081899 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed48ba8640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f2e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed493a9640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 -- 172.21.15.129:0/160081899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 msgr2=0x5641d6f16040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f16040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 -- 172.21.15.129:0/160081899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 msgr2=0x5641d6f2e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f2e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 -- 172.21.15.129:0/160081899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641d6f09780 con 0x5641d6f17240 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed48ba8640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f2e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed49baa640 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fed3c09ab10 tx=0x7fed3c014c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751d9bf2815debf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed2effd640 1 -- 172.21.15.129:0/160081899 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c0a39d0 con 0x5641d6f17240 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed2effd640 1 -- 172.21.15.129:0/160081899 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c099110 con 0x5641d6f17240 2024-09-05T01:21:23.048 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.042+0000 7fed2effd640 1 -- 172.21.15.129:0/160081899 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c09a260 con 0x5641d6f17240 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- 172.21.15.129:0/160081899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 msgr2=0x5641d6f2ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fed3c09ab10 tx=0x7fed3c014c00 comp rx=0 tx=0).stop 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- 172.21.15.129:0/160081899 shutdown_connections 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f17240 0x5641d6f2ef20 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f0c980 0x5641d6f2e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/160081899 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f16040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- 172.21.15.129:0/160081899 >> 172.21.15.129:0/160081899 conn(0x5641d6f085f0 msgr2=0x5641d6f15930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- 172.21.15.129:0/160081899 shutdown_connections 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- 172.21.15.129:0/160081899 wait complete. 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.043+0000 7fed4d9e6740 1 -- start start 2024-09-05T01:21:23.049 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 Processor -- start 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ad490 con 0x5641d6f0c980 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ae900 con 0x5641d6f17240 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed441adb60 con 0x5641d6f0c060 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed4d9e6740 10 client.0 _ll_register_callbacks cb 0x7fed400044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37962/0 (socket says 172.21.15.129:37962) 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 -- 172.21.15.129:0/2873023533 learned_addr learned my addr 172.21.15.129:0/2873023533 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed493a9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.050 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed49baa640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 -- 172.21.15.129:0/2873023533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 msgr2=0x5641d6f1b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f1b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 -- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 msgr2=0x5641d6f909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.044+0000 7fed48ba8640 1 -- 172.21.15.129:0/2873023533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641d6f1c0a0 con 0x5641d6f0c980 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed48ba8640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa60712299eed06 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34035040 con 0x5641d6f0c980 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fed34005230 con 0x5641d6f0c980 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed3c099110 con 0x5641d6f0c980 2024-09-05T01:21:23.051 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fed3405e050 con 0x5641d6f0c980 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 10 client.15975 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5641d6f91fb0 con 0x5641d6f0c980 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed3c099110 con 0x5641d6f0c980 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fed34003990 con 0x5641d6f0c980 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 10 client.15975 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 10 client.15975 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 20 client.15975 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f923b0 0x5641d6f947a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 4 client.15975 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ef602960", "release": true, "await": true}] 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.045+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ef602960", "release": true, "await": true}) -- 0x5641d6e790a0 con 0x5641d6f923b0 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.046+0000 7fed493a9640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f923b0 0x5641d6f947a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.046+0000 7fed493a9640 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f923b0 0x5641d6f947a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9249c7e68aa11dba server_cookie=2ae38fd758638a16 in_seq=0 out_seq=0 2024-09-05T01:21:23.052 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.046+0000 7fed2cff9640 10 client.15975 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:23.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed2cff9640 1 -- 172.21.15.129:0/2873023533 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5641d6e790a0 con 0x5641d6f923b0 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed2cff9640 10 client.15975 handle_command_reply: tid=0 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 client.15975 shutdown 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 msgr2=0x5641d6f1b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fed34009930 tx=0x7fed34009570 comp rx=0 tx=0).stop 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 shutdown_connections 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5641d6f923b0 0x5641d6f947a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed2cff9640 0 client.15975 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5641d6f17240 0x5641d6f909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641d6f0c980 0x5641d6f1b7c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 --2- 172.21.15.129:0/2873023533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5641d6f0c060 0x5641d6f1b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 >> 172.21.15.129:0/2873023533 conn(0x5641d6f085f0 msgr2=0x5641d6f148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.102+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 shutdown_connections 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.103+0000 7fed4d9e6740 1 -- 172.21.15.129:0/2873023533 wait complete. 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.103+0000 7fed4d9e6740 20 client.15975 trim_cache size 0 max 16384 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:21:23.105 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 20, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "ef602960": { 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 20, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 154.3, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:21:23.106 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:21:23.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 msgr2=0x7fed300a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 0x7fed300a5190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fed40009c20 tx=0x7fed40005f90 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 msgr2=0x7fed4419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00b970 tx=0x7fed3c00be30 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 shutdown_connections 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed300a2cd0 0x7fed300a5190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed44106a90 0x7fed4419cd00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed44101430 0x7fed44198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 --2- 172.21.15.129:0/1561349919 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed44100a90 0x7fed44198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 >> 172.21.15.129:0/1561349919 conn(0x7fed440fc7d0 msgr2=0x7fed44104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 shutdown_connections 2024-09-05T01:21:23.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:21:23.104+0000 7fed4b60d640 1 -- 172.21.15.129:0/1561349919 wait complete. 2024-09-05T01:21:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:22 smithi194 ceph-mon[34941]: pgmap v533: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:21:23.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:22 smithi183 ceph-mon[34589]: pgmap v533: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:21:23.943 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ef602960', total seconds elapsed: 37.7 2024-09-05T01:21:25.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 260.0 s, 286.5 tps, lat 2343.101 ms stddev 8115.079 2024-09-05T01:21:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:24 smithi194 ceph-mon[34941]: pgmap v534: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-09-05T01:21:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:24 smithi183 ceph-mon[34589]: pgmap v534: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-09-05T01:21:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:24 smithi129 ceph-mon[30979]: pgmap v534: 129 pgs: 129 active+clean; 1023 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 200 KiB/s wr, 35 op/s 2024-09-05T01:21:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/1412301113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40093f60 msgr2=0x7f4a40094360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/1412301113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40093f60 0x7f4a40094360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009900 tx=0x7f4a3c02a480 comp rx=0 tx=0).stop 2024-09-05T01:21:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/1412301113 shutdown_connections 2024-09-05T01:21:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/1412301113 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40095240 0x7f4a400956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/1412301113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40094d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/1412301113 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40093f60 0x7f4a40094360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.290+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/1412301113 >> 172.21.15.129:0/1412301113 conn(0x7f4a4008fcb0 msgr2=0x7f4a40092110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.291+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/1412301113 shutdown_connections 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.291+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/1412301113 wait complete. 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.291+0000 7f4a4d7cc640 1 Processor -- start 2024-09-05T01:21:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 -- start start 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40132420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40095240 0x7f4a40139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40097380 con 0x7f4a400948a0 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40097200 con 0x7f4a40093f60 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a4d7cc640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a40097500 con 0x7f4a40095240 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50200/0 (socket says 172.21.15.129:50200) 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.292+0000 7f4a47fff640 1 -- 172.21.15.129:0/3751100795 learned_addr learned my addr 172.21.15.129:0/3751100795 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40095240 0x7f4a40139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a477fe640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40132420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 -- 172.21.15.129:0/3751100795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40095240 msgr2=0x7f4a40139490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40095240 0x7f4a40139490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 -- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 msgr2=0x7f4a40132420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40132420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 -- 172.21.15.129:0/3751100795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a40139b00 con 0x7f4a40093f60 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a477fe640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40132420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:21:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a47fff640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a457fa640 1 -- 172.21.15.129:0/3751100795 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c03f070 con 0x7f4a40093f60 2024-09-05T01:21:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.293+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a40139d90 con 0x7f4a40093f60 2024-09-05T01:21:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.294+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4013a210 con 0x7f4a40093f60 2024-09-05T01:21:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.294+0000 7f4a457fa640 1 -- 172.21.15.129:0/3751100795 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a3c031c40 con 0x7f4a40093f60 2024-09-05T01:21:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.294+0000 7f4a457fa640 1 -- 172.21.15.129:0/3751100795 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c035030 con 0x7f4a40093f60 2024-09-05T01:21:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.297+0000 7f4a457fa640 1 -- 172.21.15.129:0/3751100795 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4a3c042070 con 0x7f4a40093f60 2024-09-05T01:21:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.297+0000 7f4a517ca740 1 Processor -- start 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a457fa640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 0x7f4a380a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 -- start start 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a457fa640 1 -- 172.21.15.129:0/3751100795 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4a3c0efcc0 con 0x7f4a40093f60 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c6540 0x5559e42c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5559e42242f0 con 0x5559e42c6540 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5559e41cc270 con 0x5559e42c14b0 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.298+0000 7f4a517ca740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5559e42202b0 con 0x5559e42c0b70 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48636/0 (socket says 172.21.15.129:48636) 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 -- 172.21.15.129:0/2445858737 learned_addr learned my addr 172.21.15.129:0/2445858737 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 0x7f4a380a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a477fe640 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c6540 0x5559e42c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 -- 172.21.15.129:0/2445858737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 msgr2=0x5559e42c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 -- 172.21.15.129:0/2445858737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c6540 msgr2=0x5559e42c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c6540 0x5559e42c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 -- 172.21.15.129:0/2445858737 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559e41cb340 con 0x5559e42c0b70 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.299+0000 7f4a47fff640 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0c47f0 tx=0x7f4a3c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82da7c5f20cee23a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.300+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 0x7f4a380a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a48067ea0 tx=0x7f4a48068bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.300+0000 7f4a327fc640 1 -- 172.21.15.129:0/2445858737 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c049070 con 0x5559e42c0b70 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.300+0000 7f4a327fc640 1 -- 172.21.15.129:0/2445858737 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a3c0c0460 con 0x5559e42c0b70 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.300+0000 7f4a327fc640 1 -- 172.21.15.129:0/2445858737 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0b8720 con 0x5559e42c0b70 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.301+0000 7f4a517ca740 1 -- 172.21.15.129:0/2445858737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 msgr2=0x5559e42c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.301+0000 7f4a517ca740 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0c47f0 tx=0x7f4a3c009260 comp rx=0 tx=0).stop 2024-09-05T01:21:26.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 -- 172.21.15.129:0/2445858737 shutdown_connections 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c6540 0x5559e42c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 --2- 172.21.15.129:0/2445858737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42c0f70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 -- 172.21.15.129:0/2445858737 >> 172.21.15.129:0/2445858737 conn(0x5559e42bc1e0 msgr2=0x5559e42bc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 -- 172.21.15.129:0/2445858737 shutdown_connections 2024-09-05T01:21:26.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.302+0000 7f4a517ca740 1 -- 172.21.15.129:0/2445858737 wait complete. 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.303+0000 7f4a517ca740 1 -- start start 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 Processor -- start 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5559e42202b0 con 0x5559e42c14b0 2024-09-05T01:21:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5559e41ad290 con 0x5559e42c6540 2024-09-05T01:21:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.304+0000 7f4a517ca740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5559e42da2e0 con 0x5559e42c0b70 2024-09-05T01:21:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a517ca740 10 client.0 _ll_register_callbacks cb 0x5559e41de140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50214/0 (socket says 172.21.15.129:50214) 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/984339543 learned_addr learned my addr 172.21.15.129:0/984339543 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a477fe640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a47fff640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/984339543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 msgr2=0x5559e42dbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42dbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/984339543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 msgr2=0x5559e42dc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42dc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/984339543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559e3f37210 con 0x5559e42c6540 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.305+0000 7f4a477fe640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a480518b0 tx=0x7f4a48063770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2005f4e84e0c2735 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4809e060 con 0x5559e42c6540 2024-09-05T01:21:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4a48078040 con 0x5559e42c6540 2024-09-05T01:21:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a3c0c0460 con 0x5559e42c6540 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a480a5050 con 0x5559e42c6540 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a517ca740 10 client.25514 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.306+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5559e4003620 con 0x5559e42c6540 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a4807cc90 con 0x5559e42c6540 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4a48098070 con 0x5559e42c6540 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 10 client.25514 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 10 client.25514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 20 client.25514 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42e7720 0x5559e42e9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 4 client.25514 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:21:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.307+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a3c0c0460 con 0x5559e42e7720 2024-09-05T01:21:26.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.308+0000 7f4a47fff640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42e7720 0x5559e42e9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.309+0000 7f4a47fff640 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42e7720 0x5559e42e9b40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9592db0b628c98cb server_cookie=14acdc756ec83153 in_seq=0 out_seq=0 2024-09-05T01:21:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.309+0000 7f4a277fe640 10 client.25514 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a277fe640 1 -- 172.21.15.129:0/984339543 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4a3c0c0460 con 0x5559e42e7720 2024-09-05T01:21:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a277fe640 10 client.25514 handle_command_reply: tid=0 2024-09-05T01:21:26.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 client.25514 shutdown 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 msgr2=0x5559e42e2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a480518b0 tx=0x7f4a48063770 comp rx=0 tx=0).stop 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 shutdown_connections 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42e7720 0x5559e42e9b40 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a277fe640 0 client.25514 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c6540 0x5559e42e2fe0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 --2- 172.21.15.129:0/984339543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42dbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 >> 172.21.15.129:0/984339543 conn(0x5559e42bc1e0 msgr2=0x5559e42bd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 shutdown_connections 2024-09-05T01:21:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 1 -- 172.21.15.129:0/984339543 wait complete. 2024-09-05T01:21:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.310+0000 7f4a517ca740 20 client.25514 trim_cache size 0 max 16384 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.338+0000 7f4a517ca740 1 Processor -- start 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.338+0000 7f4a517ca740 1 -- start start 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42bc1e0 0x5559e42c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4013eaf0 con 0x5559e42bc1e0 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4013c3c0 con 0x5559e42c14b0 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a4cfcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a4cfcb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50220/0 (socket says 172.21.15.129:50220) 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/1583509063 learned_addr learned my addr 172.21.15.129:0/1583509063 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.339+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a400aa6d0 con 0x5559e42c0b70 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.340+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/1583509063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 msgr2=0x5559e42e2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.340+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.340+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/1583509063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42bc1e0 msgr2=0x5559e42c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.340+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42bc1e0 0x5559e42c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.340+0000 7f4a4cfcb640 1 -- 172.21.15.129:0/1583509063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559e42ba7b0 con 0x5559e42c14b0 2024-09-05T01:21:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.342+0000 7f4a4cfcb640 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a48051790 tx=0x7f4a48068340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97aec499c72209a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.343+0000 7f4a31ffb640 1 -- 172.21.15.129:0/1583509063 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4809e040 con 0x5559e42c14b0 2024-09-05T01:21:26.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.343+0000 7f4a31ffb640 1 -- 172.21.15.129:0/1583509063 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a48080c80 con 0x5559e42c14b0 2024-09-05T01:21:26.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.343+0000 7f4a31ffb640 1 -- 172.21.15.129:0/1583509063 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48098e90 con 0x5559e42c14b0 2024-09-05T01:21:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.344+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 msgr2=0x5559e42c2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.344+0000 7f4a517ca740 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4a48051790 tx=0x7f4a48068340 comp rx=0 tx=0).stop 2024-09-05T01:21:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.345+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 shutdown_connections 2024-09-05T01:21:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.345+0000 7f4a517ca740 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c14b0 0x5559e42c2c10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.345+0000 7f4a517ca740 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42c0b70 0x5559e42e2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.345+0000 7f4a517ca740 1 --2- 172.21.15.129:0/1583509063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42bc1e0 0x5559e42c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.345+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 >> 172.21.15.129:0/1583509063 conn(0x5559e42c60d0 msgr2=0x5559e42dbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.349+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 shutdown_connections 2024-09-05T01:21:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.351+0000 7f4a517ca740 1 -- 172.21.15.129:0/1583509063 wait complete. 2024-09-05T01:21:26.352 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.352+0000 7f4a517ca740 1 -- start start 2024-09-05T01:21:26.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.352+0000 7f4a517ca740 1 Processor -- start 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42bc1e0 0x5559e42e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a517ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42e1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50234/0 (socket says 172.21.15.129:50234) 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a477fe640 1 -- 172.21.15.129:0/934609856 learned_addr learned my addr 172.21.15.129:0/934609856 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:26.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4013a880 con 0x5559e42c14b0 2024-09-05T01:21:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.354+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4013bcf0 con 0x5559e42c0b70 2024-09-05T01:21:26.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4013af50 con 0x5559e42bc1e0 2024-09-05T01:21:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 -- 172.21.15.129:0/934609856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42bc1e0 msgr2=0x5559e42e0430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:21:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42bc1e0 0x5559e42e0430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 -- 172.21.15.129:0/934609856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 msgr2=0x5559e42e1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42e1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559e42e1f90 con 0x5559e42c0b70 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a517ca740 10 client.0 _ll_register_callbacks cb 0x7f4a48050860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a477fe640 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a34005fd0 tx=0x7f4a340138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cbb127828887676 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a48080c80 con 0x5559e42c0b70 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.355+0000 7f4a277fe640 1 -- 172.21.15.129:0/934609856 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34004220 con 0x5559e42c0b70 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.356+0000 7f4a277fe640 1 -- 172.21.15.129:0/934609856 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4a340043c0 con 0x5559e42c0b70 2024-09-05T01:21:26.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.356+0000 7f4a277fe640 1 -- 172.21.15.129:0/934609856 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a34005020 con 0x5559e42c0b70 2024-09-05T01:21:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.357+0000 7f4a517ca740 10 client.25520 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.357+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5559e42e32a0 con 0x5559e42c0b70 2024-09-05T01:21:26.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a277fe640 1 -- 172.21.15.129:0/934609856 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4a3401a070 con 0x5559e42c0b70 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 10 client.25520 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 20 client.25520 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42c6540 0x5559e43549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 4 client.25520 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.358+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4a48080c80 con 0x5559e42c6540 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.359+0000 7f4a47fff640 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42c6540 0x5559e43549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.359+0000 7f4a47fff640 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42c6540 0x5559e43549e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51bf879223be40a7 server_cookie=529f786a7cb8a9a2 in_seq=0 out_seq=0 2024-09-05T01:21:26.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.359+0000 7f4a277fe640 10 client.25520 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:26.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.391+0000 7f4a277fe640 1 -- 172.21.15.129:0/934609856 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f4a48080c80 con 0x5559e42c6540 2024-09-05T01:21:26.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.391+0000 7f4a277fe640 10 client.25520 handle_command_reply: tid=0 2024-09-05T01:21:26.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.391+0000 7f4a517ca740 1 client.25520 shutdown 2024-09-05T01:21:26.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 msgr2=0x5559e42e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4a34005fd0 tx=0x7f4a340138f0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 shutdown_connections 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5559e42c6540 0x5559e43549e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559e42c14b0 0x5559e42e1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5559e42c0b70 0x5559e42e0970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 --2- 172.21.15.129:0/934609856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5559e42bc1e0 0x5559e42e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 >> 172.21.15.129:0/934609856 conn(0x5559e42c60d0 msgr2=0x5559e42dbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a277fe640 0 client.25520 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 shutdown_connections 2024-09-05T01:21:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.392+0000 7f4a517ca740 1 -- 172.21.15.129:0/934609856 wait complete. 2024-09-05T01:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.393+0000 7f4a517ca740 20 client.25520 trim_cache size 0 max 16384 2024-09-05T01:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:21:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:21:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.394+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 msgr2=0x7f4a380a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.394+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 0x7f4a380a50d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4a48067ea0 tx=0x7f4a48068bc0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.394+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 msgr2=0x7f4a40131ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.394+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c00 tx=0x7f4a3c003c10 comp rx=0 tx=0).stop 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 shutdown_connections 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4a380a2c10 0x7f4a380a50d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4a40095240 0x7f4a40139490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4a400948a0 0x7f4a40132420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 --2- 172.21.15.129:0/3751100795 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4a40093f60 0x7f4a40131ee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 >> 172.21.15.129:0/3751100795 conn(0x7f4a4008fcb0 msgr2=0x7f4a40091770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 shutdown_connections 2024-09-05T01:21:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:26.395+0000 7f4a4d7cc640 1 -- 172.21.15.129:0/3751100795 wait complete. 2024-09-05T01:21:27.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:27 smithi194 ceph-mon[34941]: pgmap v535: 129 pgs: 129 active+clean; 1024 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 440 KiB/s wr, 61 op/s 2024-09-05T01:21:27.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:27 smithi183 ceph-mon[34589]: pgmap v535: 129 pgs: 129 active+clean; 1024 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 440 KiB/s wr, 61 op/s 2024-09-05T01:21:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:27 smithi129 ceph-mon[30979]: pgmap v535: 129 pgs: 129 active+clean; 1024 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 440 KiB/s wr, 61 op/s 2024-09-05T01:21:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:29 smithi183 ceph-mon[34589]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 MiB/s wr, 324 op/s 2024-09-05T01:21:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:29 smithi129 ceph-mon[30979]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 MiB/s wr, 324 op/s 2024-09-05T01:21:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:29 smithi194 ceph-mon[34941]: pgmap v536: 129 pgs: 129 active+clean; 1.0 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.2 MiB/s wr, 324 op/s 2024-09-05T01:21:30.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 265.0 s, 1214.5 tps, lat 88.256 ms stddev 166.839 2024-09-05T01:21:31.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:31 smithi183 ceph-mon[34589]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.3 MiB/s wr, 493 op/s 2024-09-05T01:21:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:31 smithi129 ceph-mon[30979]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.3 MiB/s wr, 493 op/s 2024-09-05T01:21:31.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:31 smithi194 ceph-mon[34941]: pgmap v537: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.3 MiB/s wr, 493 op/s 2024-09-05T01:21:33.301 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:33 smithi194 ceph-mon[34941]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 557 op/s 2024-09-05T01:21:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:33 smithi183 ceph-mon[34589]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 557 op/s 2024-09-05T01:21:33.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:33 smithi129 ceph-mon[30979]: pgmap v538: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 10 MiB/s wr, 557 op/s 2024-09-05T01:21:34.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:34 smithi183 ceph-mon[34589]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-09-05T01:21:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:34 smithi129 ceph-mon[30979]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-09-05T01:21:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:34 smithi194 ceph-mon[34941]: pgmap v539: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 797 op/s 2024-09-05T01:21:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 270.0 s, 899.6 tps, lat 99.835 ms stddev 157.110 2024-09-05T01:21:37.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:36 smithi129 ceph-mon[30979]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 21 MiB/s wr, 894 op/s 2024-09-05T01:21:37.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:36 smithi194 ceph-mon[34941]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 21 MiB/s wr, 894 op/s 2024-09-05T01:21:37.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:36 smithi183 ceph-mon[34589]: pgmap v540: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 21 MiB/s wr, 894 op/s 2024-09-05T01:21:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:38 smithi129 ceph-mon[30979]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-09-05T01:21:39.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:38 smithi194 ceph-mon[34941]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-09-05T01:21:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:38 smithi183 ceph-mon[34589]: pgmap v541: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 25 MiB/s wr, 1.08k op/s 2024-09-05T01:21:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.553+0000 7f4fd27fd640 1 -- 172.21.15.129:0/392108079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc0736c0 msgr2=0x7f4fcc119130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/392108079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc119130 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009900 tx=0x7f4fc002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 -- 172.21.15.129:0/392108079 shutdown_connections 2024-09-05T01:21:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/392108079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc119130 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/392108079 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 0x7f4fcc073180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/392108079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc109700 0x7f4fcc109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 -- 172.21.15.129:0/392108079 >> 172.21.15.129:0/392108079 conn(0x7f4fcc06ece0 msgr2=0x7f4fcc071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 -- 172.21.15.129:0/392108079 shutdown_connections 2024-09-05T01:21:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.554+0000 7f4fd27fd640 1 -- 172.21.15.129:0/392108079 wait complete. 2024-09-05T01:21:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.555+0000 7f4fd27fd640 1 Processor -- start 2024-09-05T01:21:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.555+0000 7f4fd27fd640 1 -- start start 2024-09-05T01:21:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 0x7f4fcc115850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc109700 0x7f4fcc1b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc11c460 con 0x7f4fcc109700 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46786/0 (socket says 172.21.15.129:46786) 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/3939426926 learned_addr learned my addr 172.21.15.129:0/3939426926 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 0x7f4fcc115850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc11c2e0 con 0x7f4fcc0736c0 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc11c5e0 con 0x7f4fcc072d20 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.556+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc109700 0x7f4fcc1b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/3939426926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 msgr2=0x7f4fcc115850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 0x7f4fcc115850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc109700 msgr2=0x7f4fcc1b6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc109700 0x7f4fcc1b6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/3939426926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc116430 con 0x7f4fcc0736c0 2024-09-05T01:21:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00d770 tx=0x7f4fbc00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.557+0000 7f4fc97fa640 1 -- 172.21.15.129:0/3939426926 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc017070 con 0x7f4fcc0736c0 2024-09-05T01:21:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.558+0000 7f4fc97fa640 1 -- 172.21.15.129:0/3939426926 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fbc0054d0 con 0x7f4fcc0736c0 2024-09-05T01:21:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.559+0000 7f4fc97fa640 1 -- 172.21.15.129:0/3939426926 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc004cf0 con 0x7f4fcc0736c0 2024-09-05T01:21:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.559+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc1b71d0 con 0x7f4fcc0736c0 2024-09-05T01:21:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.559+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc1b7630 con 0x7f4fcc0736c0 2024-09-05T01:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.563+0000 7f4fd567a740 1 Processor -- start 2024-09-05T01:21:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 -- start start 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2d9a0 0x557e09f2dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f33370 0x557e09f35830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e09e912f0 con 0x557e09f2d9a0 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e09e39270 con 0x557e09f2e2e0 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fd567a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557e09e8d2b0 con 0x557e09f33370 2024-09-05T01:21:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fc97fa640 1 -- 172.21.15.129:0/3939426926 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4fbc004e90 con 0x7f4fcc0736c0 2024-09-05T01:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46788/0 (socket says 172.21.15.129:46788) 2024-09-05T01:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.564+0000 7f4fcbfff640 1 -- 172.21.15.129:0/2043547508 learned_addr learned my addr 172.21.15.129:0/2043547508 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2d9a0 0x557e09f2dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f33370 0x557e09f35830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcbfff640 1 -- 172.21.15.129:0/2043547508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f33370 msgr2=0x557e09f35830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f33370 0x557e09f35830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcbfff640 1 -- 172.21.15.129:0/2043547508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2d9a0 msgr2=0x557e09f2dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2d9a0 0x557e09f2dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.565+0000 7f4fcbfff640 1 -- 172.21.15.129:0/2043547508 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e09e38340 con 0x557e09f2e2e0 2024-09-05T01:21:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40114f0 tx=0x7f4fc40119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb85832c26e1474 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fc97fa640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 0x7f4fb40a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fb27fc640 1 -- 172.21.15.129:0/2043547508 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc401e070 con 0x557e09f2e2e0 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 0x7f4fb40a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fb27fc640 1 -- 172.21.15.129:0/2043547508 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc400fad0 con 0x557e09f2e2e0 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fb27fc640 1 -- 172.21.15.129:0/2043547508 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4019b10 con 0x557e09f2e2e0 2024-09-05T01:21:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.566+0000 7f4fc97fa640 1 -- 172.21.15.129:0/3939426926 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4fbc0cf1b0 con 0x7f4fcc0736c0 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 -- 172.21.15.129:0/2043547508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 msgr2=0x557e09f32da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 0x7f4fb40a5330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00314e0 tx=0x7f4fc0004300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40114f0 tx=0x7f4fc40119b0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 -- 172.21.15.129:0/2043547508 shutdown_connections 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f33370 0x557e09f35830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f32da0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 --2- 172.21.15.129:0/2043547508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2d9a0 0x557e09f2dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.567+0000 7f4fd567a740 1 -- 172.21.15.129:0/2043547508 >> 172.21.15.129:0/2043547508 conn(0x557e09f29010 msgr2=0x557e09f29410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.568+0000 7f4fd567a740 1 -- 172.21.15.129:0/2043547508 shutdown_connections 2024-09-05T01:21:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.568+0000 7f4fd567a740 1 -- 172.21.15.129:0/2043547508 wait complete. 2024-09-05T01:21:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.568+0000 7f4fd567a740 1 -- start start 2024-09-05T01:21:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 Processor -- start 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557e09f36340 con 0x557e09f33370 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557e09e1a290 con 0x557e09f2e2e0 2024-09-05T01:21:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fd567a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x557e09f364c0 con 0x557e09f2d9a0 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46798/0 (socket says 172.21.15.129:46798) 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fcbfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f37c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:52584/0 (socket says 172.21.15.129:52584) 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.569+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/1914564774 learned_addr learned my addr 172.21.15.129:0/1914564774 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fd567a740 10 client.0 _ll_register_callbacks cb 0x557e09e4b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/1914564774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 msgr2=0x557e09f37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/1914564774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 msgr2=0x557e09f3f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 -- 172.21.15.129:0/1914564774 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e09ba4210 con 0x557e09f2e2e0 2024-09-05T01:21:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc097f60 tx=0x7f4fbc09b3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846dfe0790cb980d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.570+0000 7f4fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:21:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.571+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc099760 con 0x557e09f2e2e0 2024-09-05T01:21:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.571+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4fbc0afd40 con 0x557e09f2e2e0 2024-09-05T01:21:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.571+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc400fad0 con 0x557e09f2e2e0 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.571+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc005180 con 0x557e09f2e2e0 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 10 client.25532 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e09c70620 con 0x557e09f2e2e0 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fbc009c30 con 0x557e09f2e2e0 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4fbc0d2150 con 0x557e09f2e2e0 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 10 client.25532 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 10 client.25532 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 20 client.25532 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09f45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 4 client.25532 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:21:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.572+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4fc400fad0 con 0x557e09f43900 2024-09-05T01:21:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.574+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09f45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.575+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09f45d30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=702db217cf5c507 server_cookie=f213a3d386866015 in_seq=0 out_seq=0 2024-09-05T01:21:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.575+0000 7f4fab7fe640 10 client.25532 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.576+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1914564774 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4fc400fad0 con 0x557e09f43900 2024-09-05T01:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.576+0000 7f4fab7fe640 10 client.25532 handle_command_reply: tid=0 2024-09-05T01:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.576+0000 7f4fd567a740 1 client.25532 shutdown 2024-09-05T01:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.577+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 msgr2=0x557e09f381c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.577+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc097f60 tx=0x7f4fbc09b3f0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 shutdown_connections 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09f45d30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fab7fe640 0 client.25532 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f381c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1914564774 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.578+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 >> 172.21.15.129:0/1914564774 conn(0x557e09f29010 msgr2=0x557e09f2a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.579+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 shutdown_connections 2024-09-05T01:21:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.579+0000 7f4fd567a740 1 -- 172.21.15.129:0/1914564774 wait complete. 2024-09-05T01:21:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.579+0000 7f4fd567a740 20 client.25532 trim_cache size 0 max 16384 2024-09-05T01:21:39.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.608+0000 7f4fd567a740 1 Processor -- start 2024-09-05T01:21:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.609+0000 7f4fd567a740 1 -- start start 2024-09-05T01:21:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f280b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2e2e0 0x557e09f3f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 0x557e09f4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1c4760 con 0x557e09f2e2e0 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1c2030 con 0x557e09f33370 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd567a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1bfe80 con 0x557e09f2d9a0 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd0d9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 0x557e09f4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd0d9a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 0x557e09f4a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:46800/0 (socket says 172.21.15.129:46800) 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fd0d9a640 1 -- 172.21.15.129:0/356322627 learned_addr learned my addr 172.21.15.129:0/356322627 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f280b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.610+0000 7f4fcb7fe640 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2e2e0 0x557e09f3f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 -- 172.21.15.129:0/356322627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 msgr2=0x557e09f4a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 0x557e09f4a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 -- 172.21.15.129:0/356322627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2e2e0 msgr2=0x557e09f3f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2e2e0 0x557e09f3f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 -- 172.21.15.129:0/356322627 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e09f2e7c0 con 0x557e09f2d9a0 2024-09-05T01:21:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.611+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f280b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000f80 tx=0x7f4fc4005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=452b269b133bc75a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.612+0000 7f4fb1ffb640 1 -- 172.21.15.129:0/356322627 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4029070 con 0x557e09f2d9a0 2024-09-05T01:21:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.612+0000 7f4fb1ffb640 1 -- 172.21.15.129:0/356322627 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc400fad0 con 0x557e09f2d9a0 2024-09-05T01:21:39.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.612+0000 7f4fb1ffb640 1 -- 172.21.15.129:0/356322627 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc401e040 con 0x557e09f2d9a0 2024-09-05T01:21:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.614+0000 7f4fd567a740 1 -- 172.21.15.129:0/356322627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 msgr2=0x557e09f280b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.614+0000 7f4fd567a740 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f280b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000f80 tx=0x7f4fc4005bd0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.617+0000 7f4fd567a740 1 -- 172.21.15.129:0/356322627 shutdown_connections 2024-09-05T01:21:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.617+0000 7f4fd567a740 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f33370 0x557e09f4a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.617+0000 7f4fd567a740 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f2e2e0 0x557e09f3f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.617+0000 7f4fd567a740 1 --2- 172.21.15.129:0/356322627 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f280b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.617+0000 7f4fd567a740 1 -- 172.21.15.129:0/356322627 >> 172.21.15.129:0/356322627 conn(0x557e09f29010 msgr2=0x557e09f279a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.618+0000 7f4fd567a740 1 -- 172.21.15.129:0/356322627 shutdown_connections 2024-09-05T01:21:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.618+0000 7f4fd567a740 1 -- 172.21.15.129:0/356322627 wait complete. 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.618+0000 7f4fd567a740 1 -- start start 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 Processor -- start 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f3cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f3d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1c04f0 con 0x557e09f33370 2024-09-05T01:21:39.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1c1960 con 0x557e09f2e2e0 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1c0bc0 con 0x557e09f2d9a0 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.619+0000 7f4fd567a740 10 client.0 _ll_register_callbacks cb 0x7f4fbc00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f3d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48530/0 (socket says 172.21.15.129:48530) 2024-09-05T01:21:39.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 -- 172.21.15.129:0/1865100295 learned_addr learned my addr 172.21.15.129:0/1865100295 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 -- 172.21.15.129:0/1865100295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 msgr2=0x557e09f3cda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f3cda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 -- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 msgr2=0x557e09f3d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f3d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.620+0000 7f4fd0d9a640 1 -- 172.21.15.129:0/1865100295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e09f3e820 con 0x557e09f33370 2024-09-05T01:21:39.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.621+0000 7f4fd0d9a640 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0004940 tx=0x7f4fc002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1673015656edff2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.621+0000 7f4fab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.621+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0035050 con 0x557e09f33370 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.621+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4fc00037f0 con 0x557e09f33370 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.621+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc400fad0 con 0x557e09f33370 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4fc005b050 con 0x557e09f33370 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 10 client.15999 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:39.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557e09faea30 con 0x557e09f33370 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc400fad0 con 0x557e09f33370 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f4fc0052470 con 0x557e09f33370 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 10 client.15999 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 20 client.15999 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.622+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09fb1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.623+0000 7f4fd567a740 4 client.15999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:21:39.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.623+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e09e9ea20 con 0x557e09f43900 2024-09-05T01:21:39.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.624+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09fb1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.627+0000 7f4fcbfff640 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09fb1450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e03d523032cd525 server_cookie=799ac8cd5b80f8e2 in_seq=0 out_seq=0 2024-09-05T01:21:39.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.627+0000 7f4fab7fe640 10 client.15999 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.668+0000 7f4fab7fe640 1 -- 172.21.15.129:0/1865100295 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e09e9ea20 con 0x557e09f43900 2024-09-05T01:21:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.668+0000 7f4fab7fe640 10 client.15999 handle_command_reply: tid=0 2024-09-05T01:21:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.668+0000 7f4fd567a740 1 client.15999 shutdown 2024-09-05T01:21:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.670+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 msgr2=0x557e09f3e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.670+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0004940 tx=0x7f4fc002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 shutdown_connections 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x557e09f43900 0x557e09fb1450 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557e09f33370 0x557e09f3e210 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557e09f2e2e0 0x557e09f3d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 --2- 172.21.15.129:0/1865100295 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x557e09f2d9a0 0x557e09f3cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 >> 172.21.15.129:0/1865100295 conn(0x557e09f29010 msgr2=0x557e09f2a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fab7fe640 0 client.15999 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.671+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 shutdown_connections 2024-09-05T01:21:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.672+0000 7f4fd567a740 1 -- 172.21.15.129:0/1865100295 wait complete. 2024-09-05T01:21:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.672+0000 7f4fd567a740 20 client.15999 trim_cache size 0 max 16384 2024-09-05T01:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:21:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:21:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.679+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 msgr2=0x7f4fb40a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.679+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 0x7f4fb40a5330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00314e0 tx=0x7f4fc0004300 comp rx=0 tx=0).stop 2024-09-05T01:21:39.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.679+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 msgr2=0x7f4fcc115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.679+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00d770 tx=0x7f4fbc00dc30 comp rx=0 tx=0).stop 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 shutdown_connections 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4fb40a2e70 0x7f4fb40a5330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fcc109700 0x7f4fcc1b6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4fcc0736c0 0x7f4fcc115d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 --2- 172.21.15.129:0/3939426926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4fcc072d20 0x7f4fcc115850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.681+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 >> 172.21.15.129:0/3939426926 conn(0x7f4fcc06ece0 msgr2=0x7f4fcc071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.683+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 shutdown_connections 2024-09-05T01:21:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:39.687+0000 7f4fd27fd640 1 -- 172.21.15.129:0/3939426926 wait complete. 2024-09-05T01:21:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 275.0 s, 1009.1 tps, lat 107.139 ms stddev 183.284 2024-09-05T01:21:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:40 smithi129 ceph-mon[30979]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 959 op/s 2024-09-05T01:21:41.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:40 smithi194 ceph-mon[34941]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 959 op/s 2024-09-05T01:21:41.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:40 smithi183 ceph-mon[34589]: pgmap v542: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 959 op/s 2024-09-05T01:21:43.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:42 smithi129 ceph-mon[30979]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 21 MiB/s wr, 868 op/s 2024-09-05T01:21:43.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:42 smithi194 ceph-mon[34941]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 21 MiB/s wr, 868 op/s 2024-09-05T01:21:43.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:42 smithi183 ceph-mon[34589]: pgmap v543: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 21 MiB/s wr, 868 op/s 2024-09-05T01:21:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 280.0 s, 1051.2 tps, lat 86.037 ms stddev 127.617 2024-09-05T01:21:45.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:44 smithi129 ceph-mon[30979]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-09-05T01:21:45.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:44 smithi194 ceph-mon[34941]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-09-05T01:21:45.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:44 smithi183 ceph-mon[34589]: pgmap v544: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-09-05T01:21:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:46 smithi129 ceph-mon[30979]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-09-05T01:21:47.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:46 smithi194 ceph-mon[34941]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-09-05T01:21:47.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:46 smithi183 ceph-mon[34589]: pgmap v545: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 19 MiB/s wr, 958 op/s 2024-09-05T01:21:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:48 smithi129 ceph-mon[30979]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-09-05T01:21:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:48 smithi194 ceph-mon[34941]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-09-05T01:21:49.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:48 smithi183 ceph-mon[34589]: pgmap v546: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 1.13k op/s 2024-09-05T01:21:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 285.0 s, 1199.6 tps, lat 90.481 ms stddev 132.119 2024-09-05T01:21:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:50 smithi129 ceph-mon[30979]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-05T01:21:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:50 smithi194 ceph-mon[34941]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-05T01:21:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:50 smithi183 ceph-mon[34589]: pgmap v547: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 20 MiB/s wr, 1.05k op/s 2024-09-05T01:21:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 -- 172.21.15.129:0/2029751022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 msgr2=0x7f9568104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 --2- 172.21.15.129:0/2029751022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f9568104160 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9558009ca0 tx=0x7f955802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 -- 172.21.15.129:0/2029751022 shutdown_connections 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 --2- 172.21.15.129:0/2029751022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 0x7f956810c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 --2- 172.21.15.129:0/2029751022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 0x7f95681053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 --2- 172.21.15.129:0/2029751022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f9568104160 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 -- 172.21.15.129:0/2029751022 >> 172.21.15.129:0/2029751022 conn(0x7f95680ff4e0 msgr2=0x7f9568101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.848+0000 7f956f333640 1 -- 172.21.15.129:0/2029751022 shutdown_connections 2024-09-05T01:21:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.849+0000 7f956f333640 1 -- 172.21.15.129:0/2029751022 wait complete. 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.849+0000 7f956f333640 1 Processor -- start 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 -- start start 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 0x7f956819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 0x7f95681a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f956810f890 con 0x7f9568104f50 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f956810f710 con 0x7f9568105910 2024-09-05T01:21:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956f333640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f956810fa10 con 0x7f9568103d60 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:50860/0 (socket says 172.21.15.129:50860) 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.850+0000 7f956d0cf640 1 -- 172.21.15.129:0/3050234335 learned_addr learned my addr 172.21.15.129:0/3050234335 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 0x7f95681a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956c8ce640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 0x7f956819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d0cf640 1 -- 172.21.15.129:0/3050234335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 msgr2=0x7f95681a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d0cf640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 0x7f95681a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d0cf640 1 -- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 msgr2=0x7f956819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d0cf640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 0x7f956819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.851+0000 7f956d0cf640 1 -- 172.21.15.129:0/3050234335 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95681a5e90 con 0x7f9568103d60 2024-09-05T01:21:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f956d0cf640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9558004af0 tx=0x7f9558003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f95667fc640 1 -- 172.21.15.129:0/3050234335 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580307c0 con 0x7f9568103d60 2024-09-05T01:21:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f95667fc640 1 -- 172.21.15.129:0/3050234335 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9558030e10 con 0x7f9568103d60 2024-09-05T01:21:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f95667fc640 1 -- 172.21.15.129:0/3050234335 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955803b410 con 0x7f9568103d60 2024-09-05T01:21:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95681a6120 con 0x7f9568103d60 2024-09-05T01:21:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.852+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95681a6620 con 0x7f9568103d60 2024-09-05T01:21:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.855+0000 7f95667fc640 1 -- 172.21.15.129:0/3050234335 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f955803b5b0 con 0x7f9568103d60 2024-09-05T01:21:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.856+0000 7f95720c9740 1 Processor -- start 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.856+0000 7f95720c9740 1 -- start start 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.856+0000 7f95720c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25889da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.856+0000 7f95720c9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f2588eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.856+0000 7f95720c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f25891830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f95720c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f257ed2f0 con 0x561f258899a0 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f95720c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561f25795270 con 0x561f2588f370 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f95720c9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f257e92b0 con 0x561f2588a2e0 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f956d8d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25889da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f2588eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f956d8d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25889da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49918/0 (socket says 172.21.15.129:49918) 2024-09-05T01:21:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f956d8d0640 1 -- 172.21.15.129:0/2294950635 learned_addr learned my addr 172.21.15.129:0/2294950635 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.857+0000 7f956c8ce640 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f25891830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f95667fc640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 0x7f95500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f956c8ce640 1 -- 172.21.15.129:0/2294950635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 msgr2=0x561f2588eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f956c8ce640 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f2588eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f956c8ce640 1 -- 172.21.15.129:0/2294950635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 msgr2=0x561f25889da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f956c8ce640 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25889da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.859+0000 7f956c8ce640 1 -- 172.21.15.129:0/2294950635 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f25794340 con 0x561f2588f370 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.860+0000 7f95667fc640 1 -- 172.21.15.129:0/3050234335 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f95580f0890 con 0x7f9568103d60 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.860+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 0x7f95500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.860+0000 7f956c8ce640 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f25891830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9560009bd0 tx=0x7f9560011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d03db497c54b138 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.861+0000 7f954f7fe640 1 -- 172.21.15.129:0/2294950635 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004230 con 0x561f2588f370 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.861+0000 7f954f7fe640 1 -- 172.21.15.129:0/2294950635 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9560009e40 con 0x561f2588f370 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.861+0000 7f954f7fe640 1 -- 172.21.15.129:0/2294950635 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560004d10 con 0x561f2588f370 2024-09-05T01:21:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.862+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 0x7f95500a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f955c00ad30 tx=0x7f955c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 -- 172.21.15.129:0/2294950635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 msgr2=0x561f25891830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f25891830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9560009bd0 tx=0x7f9560011810 comp rx=0 tx=0).stop 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 -- 172.21.15.129:0/2294950635 shutdown_connections 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f25891830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f2588eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 --2- 172.21.15.129:0/2294950635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25889da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 -- 172.21.15.129:0/2294950635 >> 172.21.15.129:0/2294950635 conn(0x561f25885010 msgr2=0x561f25885410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 -- 172.21.15.129:0/2294950635 shutdown_connections 2024-09-05T01:21:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.865+0000 7f95720c9740 1 -- 172.21.15.129:0/2294950635 wait complete. 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.866+0000 7f95720c9740 1 -- start start 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.866+0000 7f95720c9740 1 Processor -- start 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f258ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956c8ce640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956c8ce640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:50882/0 (socket says 172.21.15.129:50882) 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49932/0 (socket says 172.21.15.129:49932) 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561f258a3350 con 0x561f258899a0 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561f25776290 con 0x561f2588f370 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561f258a34d0 con 0x561f2588a2e0 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f95720c9740 10 client.0 _ll_register_callbacks cb 0x561f257a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956c8ce640 1 -- 172.21.15.129:0/3680905381 learned_addr learned my addr 172.21.15.129:0/3680905381 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.867+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f258ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 -- 172.21.15.129:0/3680905381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 msgr2=0x561f258a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 -- 172.21.15.129:0/3680905381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 msgr2=0x561f258a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 -- 172.21.15.129:0/3680905381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f25500210 con 0x561f2588f370 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.868+0000 7f956d8d0640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f258ac180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f955c010c90 tx=0x7f955c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63449137c8230e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.869+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:21:52.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.869+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c00a9a0 con 0x561f2588f370 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.869+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561f257faa20 con 0x561f2588f370 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.869+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f955c01d640 con 0x561f2588f370 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.870+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f955c01d840 con 0x561f2588f370 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.870+0000 7f95720c9740 10 client.25547 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.870+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561f255cc620 con 0x561f2588f370 2024-09-05T01:21:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.870+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561f257faa20 con 0x561f2588f370 2024-09-05T01:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f955c025070 con 0x561f2588f370 2024-09-05T01:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 10 client.25547 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 20 client.25547 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f258b08f0 0x561f258b2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 4 client.25547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:21:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.871+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f257f4d70 con 0x561f258b08f0 2024-09-05T01:21:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.872+0000 7f956d0cf640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f258b08f0 0x561f258b2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.872+0000 7f954cff9640 10 client.25547 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.872+0000 7f956d0cf640 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f258b08f0 0x561f258b2d00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae2fe688dcdb2ff server_cookie=16c01fe5e3dc4379 in_seq=0 out_seq=0 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f954cff9640 1 -- 172.21.15.129:0/3680905381 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f257f4d70 con 0x561f258b08f0 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f954cff9640 10 client.25547 handle_command_reply: tid=0 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 client.25547 shutdown 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 msgr2=0x561f258ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f258ac180 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f955c010c90 tx=0x7f955c004730 comp rx=0 tx=0).stop 2024-09-05T01:21:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 shutdown_connections 2024-09-05T01:21:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f258b08f0 0x561f258b2d00 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f258ac180 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 --2- 172.21.15.129:0/3680905381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 >> 172.21.15.129:0/3680905381 conn(0x561f25885010 msgr2=0x561f25886820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.873+0000 7f954cff9640 0 client.25547 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.874+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 shutdown_connections 2024-09-05T01:21:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.874+0000 7f95720c9740 1 -- 172.21.15.129:0/3680905381 wait complete. 2024-09-05T01:21:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.874+0000 7f95720c9740 20 client.25547 trim_cache size 0 max 16384 2024-09-05T01:21:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.902+0000 7f95720c9740 1 Processor -- start 2024-09-05T01:21:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 -- start start 2024-09-05T01:21:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25885d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588a2e0 0x561f258ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 0x561f25886290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f95681aaec0 con 0x561f258899a0 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95681a8790 con 0x561f2588a2e0 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.903+0000 7f95720c9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95680741e0 con 0x561f2588f370 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25885d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25885d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49948/0 (socket says 172.21.15.129:49948) 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d0cf640 1 -- 172.21.15.129:0/2189251432 learned_addr learned my addr 172.21.15.129:0/2189251432 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 0x561f25886290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956c8ce640 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588a2e0 0x561f258ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 -- 172.21.15.129:0/2189251432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588a2e0 msgr2=0x561f258ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588a2e0 0x561f258ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 -- 172.21.15.129:0/2189251432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 msgr2=0x561f25885d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25885d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.904+0000 7f956d8d0640 1 -- 172.21.15.129:0/2189251432 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f258a47b0 con 0x561f2588f370 2024-09-05T01:21:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.905+0000 7f956d8d0640 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 0x561f25886290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f955c017040 tx=0x7f955c00afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465529f4e917ce90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.905+0000 7f954effd640 1 -- 172.21.15.129:0/2189251432 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c01b400 con 0x561f2588f370 2024-09-05T01:21:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.905+0000 7f954effd640 1 -- 172.21.15.129:0/2189251432 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955c005020 con 0x561f2588f370 2024-09-05T01:21:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.905+0000 7f954effd640 1 -- 172.21.15.129:0/2189251432 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c0276a0 con 0x561f2588f370 2024-09-05T01:21:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.907+0000 7f95720c9740 1 -- 172.21.15.129:0/2189251432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 msgr2=0x561f25886290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.907+0000 7f95720c9740 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 0x561f25886290 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f955c017040 tx=0x7f955c00afd0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.907+0000 7f95720c9740 1 -- 172.21.15.129:0/2189251432 shutdown_connections 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.907+0000 7f95720c9740 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588f370 0x561f25886290 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.908+0000 7f95720c9740 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588a2e0 0x561f258ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.908+0000 7f95720c9740 1 --2- 172.21.15.129:0/2189251432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f25885d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.908+0000 7f95720c9740 1 -- 172.21.15.129:0/2189251432 >> 172.21.15.129:0/2189251432 conn(0x561f25885010 msgr2=0x561f258b26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.908+0000 7f95720c9740 1 -- 172.21.15.129:0/2189251432 shutdown_connections 2024-09-05T01:21:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.909+0000 7f95720c9740 1 -- 172.21.15.129:0/2189251432 wait complete. 2024-09-05T01:21:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.910+0000 7f95720c9740 1 -- start start 2024-09-05T01:21:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.910+0000 7f95720c9740 1 Processor -- start 2024-09-05T01:21:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f95720c9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f95720c9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f95720c9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f259195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f956d0cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49964/0 (socket says 172.21.15.129:49964) 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f956d0cf640 1 -- 172.21.15.129:0/233358341 learned_addr learned my addr 172.21.15.129:0/233358341 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.911+0000 7f956c8ce640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f95681a6c50 con 0x561f258899a0 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95681a80c0 con 0x561f2588f370 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f95681a7320 con 0x561f2588a2e0 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f259195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f95720c9740 10 client.0 _ll_register_callbacks cb 0x7f955c00a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:21:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 -- 172.21.15.129:0/233358341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 msgr2=0x561f258a93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 -- 172.21.15.129:0/233358341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 msgr2=0x561f258a8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:21:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f258ac4a0 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f956d8d0640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f259195d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f955c027dd0 tx=0x7f955c01dc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87758456cab2dafe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.912+0000 7f954cff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c021040 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955c005020 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f955c027650 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f955c04c050 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f95720c9740 10 client.25553 fetch_fsmap learned FSMap version 30 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561f2591ab70 con 0x561f2588f370 2024-09-05T01:21:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.913+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f955c005020 con 0x561f2588f370 2024-09-05T01:21:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f955c047030 con 0x561f2588f370 2024-09-05T01:21:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 10 client.25553 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:21:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 10 client.25553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:21:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 20 client.25553 populate_metadata read hostname 'smithi129' 2024-09-05T01:21:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f2591b570 0x561f2591d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:21:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 4 client.25553 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:21:52.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.915+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f25771900 con 0x561f2591b570 2024-09-05T01:21:52.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.917+0000 7f956d0cf640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f2591b570 0x561f2591d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:21:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.919+0000 7f956d0cf640 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f2591b570 0x561f2591d990 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26471c8f74689df server_cookie=2d4025b467d6f564 in_seq=0 out_seq=0 2024-09-05T01:21:52.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.919+0000 7f954cff9640 10 client.25553 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.957+0000 7f954cff9640 1 -- 172.21.15.129:0/233358341 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f25771900 con 0x561f2591b570 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.957+0000 7f954cff9640 10 client.25553 handle_command_reply: tid=0 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.957+0000 7f95720c9740 1 client.25553 shutdown 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 msgr2=0x561f259195d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f259195d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f955c027dd0 tx=0x7f955c01dc70 comp rx=0 tx=0).stop 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 shutdown_connections 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561f2591b570 0x561f2591d990 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561f2588f370 0x561f259195d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561f2588a2e0 0x561f258a93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 --2- 172.21.15.129:0/233358341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561f258899a0 0x561f258a8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 >> 172.21.15.129:0/233358341 conn(0x561f25885010 msgr2=0x561f258839a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.958+0000 7f954cff9640 0 client.25553 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.959+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 shutdown_connections 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.959+0000 7f95720c9740 1 -- 172.21.15.129:0/233358341 wait complete. 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.959+0000 7f95720c9740 20 client.25553 trim_cache size 0 max 16384 2024-09-05T01:21:52.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:21:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:21:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:21:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:21:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.962+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 msgr2=0x7f95500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.962+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 0x7f95500a50f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f955c00ad30 tx=0x7f955c00a250 comp rx=0 tx=0).stop 2024-09-05T01:21:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.962+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 msgr2=0x7f956819e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:21:52.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.962+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9558004af0 tx=0x7f9558003b80 comp rx=0 tx=0).stop 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 shutdown_connections 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f95500a2c30 0x7f95500a50f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9568105910 0x7f95681a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9568104f50 0x7f956819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 --2- 172.21.15.129:0/3050234335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9568103d60 0x7f956819e270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:21:52.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.963+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 >> 172.21.15.129:0/3050234335 conn(0x7f95680ff4e0 msgr2=0x7f9568078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:21:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.964+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 shutdown_connections 2024-09-05T01:21:52.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:21:52.964+0000 7f956f333640 1 -- 172.21.15.129:0/3050234335 wait complete. 2024-09-05T01:21:53.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:52 smithi129 ceph-mon[30979]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 20 MiB/s wr, 980 op/s 2024-09-05T01:21:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:52 smithi194 ceph-mon[34941]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 20 MiB/s wr, 980 op/s 2024-09-05T01:21:53.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:52 smithi183 ceph-mon[34589]: pgmap v548: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 20 MiB/s wr, 980 op/s 2024-09-05T01:21:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 290.0 s, 992.4 tps, lat 105.519 ms stddev 155.582 2024-09-05T01:21:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:54 smithi129 ceph-mon[30979]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 23 MiB/s wr, 1.12k op/s 2024-09-05T01:21:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:54 smithi194 ceph-mon[34941]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 23 MiB/s wr, 1.12k op/s 2024-09-05T01:21:55.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:54 smithi183 ceph-mon[34589]: pgmap v549: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 23 MiB/s wr, 1.12k op/s 2024-09-05T01:21:57.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:56 smithi129 ceph-mon[30979]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 18 MiB/s wr, 967 op/s 2024-09-05T01:21:57.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:56 smithi194 ceph-mon[34941]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 18 MiB/s wr, 967 op/s 2024-09-05T01:21:57.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:56 smithi183 ceph-mon[34589]: pgmap v550: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 18 MiB/s wr, 967 op/s 2024-09-05T01:21:59.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:21:58 smithi129 ceph-mon[30979]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-05T01:21:59.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:21:58 smithi194 ceph-mon[34941]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-05T01:21:59.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:21:58 smithi183 ceph-mon[34589]: pgmap v551: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 21 MiB/s wr, 1.13k op/s 2024-09-05T01:22:00.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 295.0 s, 1046.6 tps, lat 92.255 ms stddev 102.316 2024-09-05T01:22:01.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:00 smithi194 ceph-mon[34941]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 17 MiB/s wr, 977 op/s 2024-09-05T01:22:01.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:00 smithi129 ceph-mon[30979]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 17 MiB/s wr, 977 op/s 2024-09-05T01:22:01.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:00 smithi183 ceph-mon[34589]: pgmap v552: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 17 MiB/s wr, 977 op/s 2024-09-05T01:22:03.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:02 smithi194 ceph-mon[34941]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 923 op/s 2024-09-05T01:22:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:02 smithi183 ceph-mon[34589]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 923 op/s 2024-09-05T01:22:03.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:02 smithi129 ceph-mon[30979]: pgmap v553: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 923 op/s 2024-09-05T01:22:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 300.0 s, 972.7 tps, lat 106.579 ms stddev 156.976 2024-09-05T01:22:05.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:05 smithi183 ceph-mon[34589]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 1.13k op/s 2024-09-05T01:22:05.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:05 smithi129 ceph-mon[30979]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 1.13k op/s 2024-09-05T01:22:05.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:05 smithi194 ceph-mon[34941]: pgmap v554: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 1.13k op/s 2024-09-05T01:22:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.117+0000 7f887e8cf640 1 -- 172.21.15.129:0/1663363926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 msgr2=0x7f8878100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.117+0000 7f887e8cf640 1 --2- 172.21.15.129:0/1663363926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f8878100ff0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8868009860 tx=0x7f886802a340 comp rx=0 tx=0).stop 2024-09-05T01:22:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 -- 172.21.15.129:0/1663363926 shutdown_connections 2024-09-05T01:22:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 --2- 172.21.15.129:0/1663363926 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 --2- 172.21.15.129:0/1663363926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f8878100ff0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 --2- 172.21.15.129:0/1663363926 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f8878100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 -- 172.21.15.129:0/1663363926 >> 172.21.15.129:0/1663363926 conn(0x7f88780fbfa0 msgr2=0x7f88780fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 -- 172.21.15.129:0/1663363926 shutdown_connections 2024-09-05T01:22:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.118+0000 7f887e8cf640 1 -- 172.21.15.129:0/1663363926 wait complete. 2024-09-05T01:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.119+0000 7f887e8cf640 1 Processor -- start 2024-09-05T01:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.119+0000 7f887e8cf640 1 -- start start 2024-09-05T01:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f887819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887d8cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887d8cd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38376/0 (socket says 172.21.15.129:38376) 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887d8cd640 1 -- 172.21.15.129:0/4221973133 learned_addr learned my addr 172.21.15.129:0/4221973133 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f887810f6a0 con 0x7f8878100b90 2024-09-05T01:22:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f887810f520 con 0x7f8878101530 2024-09-05T01:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e0ce640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.120+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f887810f820 con 0x7f8878100250 2024-09-05T01:22:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f887819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 -- 172.21.15.129:0/4221973133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 msgr2=0x7f887819e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 -- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 msgr2=0x7f88781a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d0cc640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781a5ca0 con 0x7f8878100b90 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887e0ce640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.121+0000 7f887d8cd640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.122+0000 7f887d0cc640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f887819e5c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8868009780 tx=0x7f8868032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.122+0000 7f886effd640 1 -- 172.21.15.129:0/4221973133 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868040020 con 0x7f8878100b90 2024-09-05T01:22:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.122+0000 7f886effd640 1 -- 172.21.15.129:0/4221973133 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8868038040 con 0x7f8878100b90 2024-09-05T01:22:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.122+0000 7f886effd640 1 -- 172.21.15.129:0/4221973133 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868037070 con 0x7f8878100b90 2024-09-05T01:22:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.123+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88781a5f30 con 0x7f8878100b90 2024-09-05T01:22:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.123+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a6340 con 0x7f8878100b90 2024-09-05T01:22:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.126+0000 7f88828cc740 1 Processor -- start 2024-09-05T01:22:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.128+0000 7f886effd640 1 -- 172.21.15.129:0/4221973133 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8868043020 con 0x7f8878100b90 2024-09-05T01:22:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.128+0000 7f88828cc740 1 -- start start 2024-09-05T01:22:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774981f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774984a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587748e02f0 con 0x558774982540 2024-09-05T01:22:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558774888270 con 0x55877497cb70 2024-09-05T01:22:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f88828cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5587748dc2b0 con 0x55877497d4b0 2024-09-05T01:22:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.129+0000 7f887e0ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53212/0 (socket says 172.21.15.129:53212) 2024-09-05T01:22:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 -- 172.21.15.129:0/3106804864 learned_addr learned my addr 172.21.15.129:0/3106804864 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f886effd640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 0x7f88640a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774984a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 -- 172.21.15.129:0/3106804864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 msgr2=0x558774981f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774981f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 -- 172.21.15.129:0/3106804864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 msgr2=0x558774984a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774984a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.130+0000 7f887e0ce640 1 -- 172.21.15.129:0/3106804864 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558774887340 con 0x55877497cb70 2024-09-05T01:22:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.131+0000 7f886effd640 1 -- 172.21.15.129:0/4221973133 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f88680eeed0 con 0x7f8878100b90 2024-09-05T01:22:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.132+0000 7f887e0ce640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 0x7f88640a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.132+0000 7f887e0ce640 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f88740116a0 tx=0x7f8874011b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a9b82cca50b45a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.132+0000 7f8863fff640 1 -- 172.21.15.129:0/3106804864 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874017050 con 0x55877497cb70 2024-09-05T01:22:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.132+0000 7f8863fff640 1 -- 172.21.15.129:0/3106804864 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f887400fdc0 con 0x55877497cb70 2024-09-05T01:22:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.132+0000 7f8863fff640 1 -- 172.21.15.129:0/3106804864 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887401a830 con 0x55877497cb70 2024-09-05T01:22:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.133+0000 7f88828cc740 1 -- 172.21.15.129:0/3106804864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 msgr2=0x55877497cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.133+0000 7f88828cc740 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f88740116a0 tx=0x7f8874011b60 comp rx=0 tx=0).stop 2024-09-05T01:22:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f887e0ce640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 0x7f88640a51d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8874012010 tx=0x7f887401af40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f88828cc740 1 -- 172.21.15.129:0/3106804864 shutdown_connections 2024-09-05T01:22:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f88828cc740 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774984a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f88828cc740 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774981f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f88828cc740 1 --2- 172.21.15.129:0/3106804864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877497cf70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.134+0000 7f88828cc740 1 -- 172.21.15.129:0/3106804864 >> 172.21.15.129:0/3106804864 conn(0x5587749781e0 msgr2=0x5587749785e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.135+0000 7f88828cc740 1 -- 172.21.15.129:0/3106804864 shutdown_connections 2024-09-05T01:22:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.135+0000 7f88828cc740 1 -- 172.21.15.129:0/3106804864 wait complete. 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.135+0000 7f88828cc740 1 -- start start 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.135+0000 7f88828cc740 1 Processor -- start 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774998320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558774982540 0x55877499f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558774985510 con 0x55877497cb70 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558774869290 con 0x558774982540 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558774985690 con 0x55877497d4b0 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.136+0000 7f88828cc740 10 client.0 _ll_register_callbacks cb 0x55877489a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.137+0000 7f887d8cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.137+0000 7f887d8cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33784/0 (socket says 172.21.15.129:33784) 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.137+0000 7f887d8cd640 1 -- 172.21.15.129:0/1988004118 learned_addr learned my addr 172.21.15.129:0/1988004118 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 -- 172.21.15.129:0/1988004118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 msgr2=0x558774998320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774998320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 -- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558774982540 msgr2=0x55877499f360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558774982540 0x55877499f360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 -- 172.21.15.129:0/1988004118 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587745f3210 con 0x55877497cb70 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.138+0000 7f887d8cd640 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f887000ef70 tx=0x7f887000d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4322e41cc2d1d133 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.139+0000 7f88617fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.139+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8870016790 con 0x55877497cb70 2024-09-05T01:22:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.139+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f887400fdc0 con 0x55877497cb70 2024-09-05T01:22:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.141+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f8870016930 con 0x55877497cb70 2024-09-05T01:22:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.142+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8870025050 con 0x55877497cb70 2024-09-05T01:22:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.143+0000 7f88828cc740 10 client.16026 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.143+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587746bf620 con 0x55877497cb70 2024-09-05T01:22:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.146+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f887400fdc0 con 0x55877497cb70 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.146+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f887000b040 con 0x55877497cb70 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.147+0000 7f88828cc740 10 client.16026 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.148+0000 7f88828cc740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.148+0000 7f88828cc740 20 client.16026 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.148+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774989890 0x55877498bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.148+0000 7f88828cc740 4 client.16026 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.148+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587748eda20 con 0x558774989890 2024-09-05T01:22:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.149+0000 7f887d0cc640 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774989890 0x55877498bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.150+0000 7f887d0cc640 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774989890 0x55877498bd10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e4590bf79547ca server_cookie=1ba573f1a828b7f7 in_seq=0 out_seq=0 2024-09-05T01:22:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.150+0000 7f88617fa640 10 client.16026 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.151+0000 7f88617fa640 1 -- 172.21.15.129:0/1988004118 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587748eda20 con 0x558774989890 2024-09-05T01:22:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.151+0000 7f88617fa640 10 client.16026 handle_command_reply: tid=0 2024-09-05T01:22:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.151+0000 7f88828cc740 1 client.16026 shutdown 2024-09-05T01:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.157+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 msgr2=0x558774997de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.157+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f887000ef70 tx=0x7f887000d870 comp rx=0 tx=0).stop 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 shutdown_connections 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774989890 0x55877498bd10 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558774982540 0x55877499f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x558774998320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 --2- 172.21.15.129:0/1988004118 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497cb70 0x558774997de0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 >> 172.21.15.129:0/1988004118 conn(0x5587749781e0 msgr2=0x5587749799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88617fa640 0 client.16026 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.159+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 shutdown_connections 2024-09-05T01:22:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.160+0000 7f88828cc740 1 -- 172.21.15.129:0/1988004118 wait complete. 2024-09-05T01:22:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.160+0000 7f88828cc740 20 client.16026 trim_cache size 0 max 16384 2024-09-05T01:22:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.194+0000 7f88828cc740 1 Processor -- start 2024-09-05T01:22:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 -- start start 2024-09-05T01:22:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x5587749785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774978b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88781aadd0 con 0x558774982540 2024-09-05T01:22:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88781a86a0 con 0x55877497cb70 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f88828cc740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88781a6630 con 0x55877497d4b0 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f887d0cc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.195+0000 7f887e0ce640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774978b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:38380/0 (socket says 172.21.15.129:38380) 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 -- 172.21.15.129:0/3902419322 learned_addr learned my addr 172.21.15.129:0/3902419322 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d8cd640 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x5587749785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 -- 172.21.15.129:0/3902419322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 msgr2=0x5587749785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x5587749785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 -- 172.21.15.129:0/3902419322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 msgr2=0x558774978b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774978b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 -- 172.21.15.129:0/3902419322 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558774978000 con 0x55877497d4b0 2024-09-05T01:22:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8868000c00 tx=0x7f88680b76e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa3cf30b861c256 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f88637fe640 1 -- 172.21.15.129:0/3902419322 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680cf070 con 0x55877497d4b0 2024-09-05T01:22:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f88637fe640 1 -- 172.21.15.129:0/3902419322 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f886800b0b0 con 0x55877497d4b0 2024-09-05T01:22:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.196+0000 7f88637fe640 1 -- 172.21.15.129:0/3902419322 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680b9740 con 0x55877497d4b0 2024-09-05T01:22:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 -- 172.21.15.129:0/3902419322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 msgr2=0x55877499f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8868000c00 tx=0x7f88680b76e0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 -- 172.21.15.129:0/3902419322 shutdown_connections 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558774982540 0x558774978b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55877497d4b0 0x55877499f320 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 --2- 172.21.15.129:0/3902419322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x5587749785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.197+0000 7f88828cc740 1 -- 172.21.15.129:0/3902419322 >> 172.21.15.129:0/3902419322 conn(0x5587749781e0 msgr2=0x5587749772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.198+0000 7f88828cc740 1 -- 172.21.15.129:0/3902419322 shutdown_connections 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.198+0000 7f88828cc740 1 -- 172.21.15.129:0/3902419322 wait complete. 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.199+0000 7f88828cc740 1 -- start start 2024-09-05T01:22:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.200+0000 7f88828cc740 1 Processor -- start 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f88828cc740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877499ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f88828cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f88828cc740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 0x55877499dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f88828cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88781a6b60 con 0x55877497d4b0 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f887d0cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f887d0cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:33800/0 (socket says 172.21.15.129:33800) 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f887d0cc640 1 -- 172.21.15.129:0/3353206840 learned_addr learned my addr 172.21.15.129:0/3353206840 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.201+0000 7f887d8cd640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877499ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887e0ce640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 0x55877499dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f88781a7fd0 con 0x55877497cb70 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f88781a7230 con 0x558774982540 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f88828cc740 10 client.0 _ll_register_callbacks cb 0x7f8870009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d0cc640 1 -- 172.21.15.129:0/3353206840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 msgr2=0x55877499dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 0x55877499dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d0cc640 1 -- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 msgr2=0x55877499ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877499ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d0cc640 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55877499e040 con 0x55877497d4b0 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887e0ce640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 0x55877499dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.202+0000 7f887d8cd640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877499ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f887d0cc640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f886802a380 tx=0x7f88680057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f936666ee02a067 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88617fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680cf070 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f88680b7ab0 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886800b0b0 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f88680c4020 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88828cc740 10 client.16032 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.203+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558774a1fab0 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88680bbb40 con 0x55877497d4b0 2024-09-05T01:22:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f88680c3b90 con 0x55877497d4b0 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 10 client.16032 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 20 client.16032 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774a203f0 0x558774a22810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 4 client.16032 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f886800b0b0 con 0x558774a203f0 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.204+0000 7f887d8cd640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774a203f0 0x558774a22810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.205+0000 7f88617fa640 10 client.16032 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:06.207 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.206+0000 7f887d8cd640 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774a203f0 0x558774a22810 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66831a2cb84c07c2 server_cookie=8063796103e00847 in_seq=0 out_seq=0 2024-09-05T01:22:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.233+0000 7f88617fa640 1 -- 172.21.15.129:0/3353206840 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f886800b0b0 con 0x558774a203f0 2024-09-05T01:22:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.233+0000 7f88617fa640 10 client.16032 handle_command_reply: tid=0 2024-09-05T01:22:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.233+0000 7f88828cc740 1 client.16032 shutdown 2024-09-05T01:22:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.234+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 msgr2=0x55877499d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.234+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f886802a380 tx=0x7f88680057e0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 shutdown_connections 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558774a203f0 0x558774a22810 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558774982540 0x55877499dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55877497d4b0 0x55877499d590 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 --2- 172.21.15.129:0/3353206840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55877497cb70 0x55877499ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 >> 172.21.15.129:0/3353206840 conn(0x5587749781e0 msgr2=0x558774996600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88617fa640 0 client.16032 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 shutdown_connections 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 1 -- 172.21.15.129:0/3353206840 wait complete. 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.235+0000 7f88828cc740 20 client.16032 trim_cache size 0 max 16384 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:22:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:22:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.237+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 msgr2=0x7f88640a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.237+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 0x7f88640a51d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8874012010 tx=0x7f887401af40 comp rx=0 tx=0).stop 2024-09-05T01:22:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 msgr2=0x7f887819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f887819e5c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8868009780 tx=0x7f8868032e80 comp rx=0 tx=0).stop 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 shutdown_connections 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f88640a2d10 0x7f88640a51d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8878101530 0x7f88781a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8878100b90 0x7f887819e5c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 --2- 172.21.15.129:0/4221973133 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8878100250 0x7f887819e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 >> 172.21.15.129:0/4221973133 conn(0x7f88780fbfa0 msgr2=0x7f88780fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.238+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 shutdown_connections 2024-09-05T01:22:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:06.239+0000 7f887e8cf640 1 -- 172.21.15.129:0/4221973133 wait complete. 2024-09-05T01:22:07.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:07 smithi183 ceph-mon[34589]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-05T01:22:07.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:07 smithi129 ceph-mon[30979]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-05T01:22:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:07 smithi194 ceph-mon[34941]: pgmap v555: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-09-05T01:22:09.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:09 smithi183 ceph-mon[34589]: pgmap v556: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-09-05T01:22:09.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:09 smithi129 ceph-mon[30979]: pgmap v556: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-09-05T01:22:09.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:09 smithi194 ceph-mon[34941]: pgmap v556: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 19 MiB/s wr, 1.16k op/s 2024-09-05T01:22:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 305.0 s, 1064.6 tps, lat 93.497 ms stddev 104.070 2024-09-05T01:22:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:11 smithi183 ceph-mon[34589]: pgmap v557: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:22:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:11 smithi129 ceph-mon[30979]: pgmap v557: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:22:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:11 smithi194 ceph-mon[34941]: pgmap v557: 129 pgs: 129 active+clean; 1.2 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:22:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:13 smithi183 ceph-mon[34589]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 966 op/s 2024-09-05T01:22:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:13 smithi129 ceph-mon[30979]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 966 op/s 2024-09-05T01:22:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:13 smithi194 ceph-mon[34941]: pgmap v558: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 966 op/s 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:22:14.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:22:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:22:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:22:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:22:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:22:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:22:14.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:22:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 310.0 s, 1040.6 tps, lat 95.944 ms stddev 104.106 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:22:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:15 smithi183 ceph-mon[34589]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-09-05T01:22:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:22:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:22:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:22:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:22:15.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:22:15.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:22:15.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:15 smithi129 ceph-mon[30979]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:22:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:22:15.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:15 smithi194 ceph-mon[34941]: pgmap v559: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 18 MiB/s wr, 1.15k op/s 2024-09-05T01:22:17.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:17 smithi183 ceph-mon[34589]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 997 op/s 2024-09-05T01:22:17.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:17 smithi129 ceph-mon[30979]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 997 op/s 2024-09-05T01:22:17.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:17 smithi194 ceph-mon[34941]: pgmap v560: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 15 MiB/s wr, 997 op/s 2024-09-05T01:22:19.294 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:19 smithi129 ceph-mon[30979]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-09-05T01:22:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 -- 172.21.15.129:0/755903611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a6030 msgr2=0x7f39a40aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 --2- 172.21.15.129:0/755903611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a6030 0x7f39a40aa8d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f399c009900 tx=0x7f399c02a480 comp rx=0 tx=0).stop 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 -- 172.21.15.129:0/755903611 shutdown_connections 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 --2- 172.21.15.129:0/755903611 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a6030 0x7f39a40aa8d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 --2- 172.21.15.129:0/755903611 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 --2- 172.21.15.129:0/755903611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a4540 0x7f39a40a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.386+0000 7f39abfff640 1 -- 172.21.15.129:0/755903611 >> 172.21.15.129:0/755903611 conn(0x7f39a409fba0 msgr2=0x7f39a40a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.387+0000 7f39abfff640 1 -- 172.21.15.129:0/755903611 shutdown_connections 2024-09-05T01:22:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.387+0000 7f39abfff640 1 -- 172.21.15.129:0/755903611 wait complete. 2024-09-05T01:22:19.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.387+0000 7f39abfff640 1 Processor -- start 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 -- start start 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 0x7f39a40b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 0x7f39a4152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40b88a0 con 0x7f39a40a5670 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40b8720 con 0x7f39a40a6030 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39abfff640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f39a40b8a20 con 0x7f39a40a4540 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 0x7f39a40b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 0x7f39a40b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:49022/0 (socket says 172.21.15.129:49022) 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39aaffd640 1 -- 172.21.15.129:0/4208143583 learned_addr learned my addr 172.21.15.129:0/4208143583 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:19.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.388+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 0x7f39a4152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 -- 172.21.15.129:0/4208143583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 msgr2=0x7f39a40b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 0x7f39a40b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 -- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 msgr2=0x7f39a40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 -- 172.21.15.129:0/4208143583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a41534f0 con 0x7f39a40a6030 2024-09-05T01:22:19.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 0x7f39a4152fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f399c02aa70 tx=0x7f399c003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.389+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.390+0000 7f3993fff640 1 -- 172.21.15.129:0/4208143583 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c03f070 con 0x7f39a40a6030 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.390+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a41536c0 con 0x7f39a40a6030 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.390+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a4153ac0 con 0x7f39a40a6030 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.390+0000 7f3993fff640 1 -- 172.21.15.129:0/4208143583 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f399c031e40 con 0x7f39a40a6030 2024-09-05T01:22:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.390+0000 7f3993fff640 1 -- 172.21.15.129:0/4208143583 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c035030 con 0x7f39a40a6030 2024-09-05T01:22:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.394+0000 7f3993fff640 1 -- 172.21.15.129:0/4208143583 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f399c042070 con 0x7f39a40a6030 2024-09-05T01:22:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.394+0000 7f3993fff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 0x7f39940a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.395+0000 7f3993fff640 1 -- 172.21.15.129:0/4208143583 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f399c0f0be0 con 0x7f39a40a6030 2024-09-05T01:22:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.395+0000 7f39aaffd640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 0x7f39940a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:19 smithi183 ceph-mon[34589]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-09-05T01:22:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.427+0000 7f39aaffd640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 0x7f39940a51a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0518b0 tx=0x7f39ac0505a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 Processor -- start 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 -- start start 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10e5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b10e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5598b10442f0 con 0x5598b10e6540 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598b0fec270 con 0x5598b10e0b70 2024-09-05T01:22:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39b4616740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5598b10402b0 con 0x5598b10e14b0 2024-09-05T01:22:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34758/0 (socket says 172.21.15.129:34758) 2024-09-05T01:22:19.431 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39aaffd640 1 -- 172.21.15.129:0/1213396042 learned_addr learned my addr 172.21.15.129:0/1213396042 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.429+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10e5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b10e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 -- 172.21.15.129:0/1213396042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 msgr2=0x5598b10e5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10e5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 -- 172.21.15.129:0/1213396042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 msgr2=0x5598b10e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b10e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 -- 172.21.15.129:0/1213396042 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b0feb340 con 0x5598b10e0b70 2024-09-05T01:22:19.432 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.430+0000 7f39aaffd640 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0637b0 tx=0x7f39ac078000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f11368fe9e62934 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.431+0000 7f39917fa640 1 -- 172.21.15.129:0/1213396042 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac076dc0 con 0x5598b10e0b70 2024-09-05T01:22:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.431+0000 7f39917fa640 1 -- 172.21.15.129:0/1213396042 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39ac086030 con 0x5598b10e0b70 2024-09-05T01:22:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.431+0000 7f39917fa640 1 -- 172.21.15.129:0/1213396042 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac082650 con 0x5598b10e0b70 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 -- 172.21.15.129:0/1213396042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 msgr2=0x5598b10e0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0637b0 tx=0x7f39ac078000 comp rx=0 tx=0).stop 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 -- 172.21.15.129:0/1213396042 shutdown_connections 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b10e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10e5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 --2- 172.21.15.129:0/1213396042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10e0f70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.433+0000 7f39b4616740 1 -- 172.21.15.129:0/1213396042 >> 172.21.15.129:0/1213396042 conn(0x5598b10dc1e0 msgr2=0x5598b10dc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.434+0000 7f39b4616740 1 -- 172.21.15.129:0/1213396042 shutdown_connections 2024-09-05T01:22:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.434+0000 7f39b4616740 1 -- 172.21.15.129:0/1213396042 wait complete. 2024-09-05T01:22:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.435+0000 7f39b4616740 1 -- start start 2024-09-05T01:22:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.435+0000 7f39b4616740 1 Processor -- start 2024-09-05T01:22:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39b4616740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39b4616740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:49044/0 (socket says 172.21.15.129:49044) 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39aaffd640 1 -- 172.21.15.129:0/3262809930 learned_addr learned my addr 172.21.15.129:0/3262809930 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e6540 0x5598b10f2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5598b10e9510 con 0x5598b10e14b0 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.436+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5598b0fcd290 con 0x5598b10e6540 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e6540 0x5598b10f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5598b10e9690 con 0x5598b10e0b70 2024-09-05T01:22:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39b4616740 10 client.0 _ll_register_callbacks cb 0x5598b0ffe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 -- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e6540 msgr2=0x5598b10f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e6540 0x5598b10f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 -- 172.21.15.129:0/3262809930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 msgr2=0x5598b10eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b0d57210 con 0x5598b10e0b70 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.437+0000 7f39aaffd640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06adf0 tx=0x7f39ac06ae20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b9e647889331250 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39ac086030 con 0x5598b10e0b70 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f398eff5640 10 client.25534 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f398eff5640 1 -- 172.21.15.129:0/3262809930 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac076d80 con 0x5598b10e0b70 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f398eff5640 1 -- 172.21.15.129:0/3262809930 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f39ac081410 con 0x5598b10e0b70 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f398eff5640 1 -- 172.21.15.129:0/3262809930 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f39ac081610 con 0x5598b10e0b70 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f39b4616740 10 client.25534 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.438+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5598b0e23620 con 0x5598b10e0b70 2024-09-05T01:22:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.439+0000 7f398eff5640 1 -- 172.21.15.129:0/3262809930 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f39ac0817b0 con 0x5598b10e0b70 2024-09-05T01:22:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.439+0000 7f39b4616740 10 client.25534 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.439+0000 7f39b4616740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.439+0000 7f39b4616740 20 client.25534 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.440+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b10f6980 0x5598b10f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.440+0000 7f39b4616740 4 client.25534 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.440+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f39ac086030 con 0x5598b10f6980 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.440+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b10f6980 0x5598b10f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.441+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b10f6980 0x5598b10f8d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eea02d8ff1a1a288 server_cookie=40edb329e7654fb1 in_seq=0 out_seq=0 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.441+0000 7f398eff5640 10 client.25534 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.441+0000 7f398eff5640 1 -- 172.21.15.129:0/3262809930 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f39ac086030 con 0x5598b10f6980 2024-09-05T01:22:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.442+0000 7f398eff5640 10 client.25534 handle_command_reply: tid=0 2024-09-05T01:22:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.442+0000 7f39b4616740 1 client.25534 shutdown 2024-09-05T01:22:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.443+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 msgr2=0x5598b10eadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.443+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06adf0 tx=0x7f39ac06ae20 comp rx=0 tx=0).stop 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.443+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 shutdown_connections 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.443+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b10f6980 0x5598b10f8d70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e6540 0x5598b10f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 --2- 172.21.15.129:0/3262809930 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e0b70 0x5598b10eadf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 >> 172.21.15.129:0/3262809930 conn(0x5598b10dc1e0 msgr2=0x5598b10dd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f398eff5640 0 client.25534 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 shutdown_connections 2024-09-05T01:22:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 1 -- 172.21.15.129:0/3262809930 wait complete. 2024-09-05T01:22:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.444+0000 7f39b4616740 20 client.25534 trim_cache size 0 max 16384 2024-09-05T01:22:19.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 Processor -- start 2024-09-05T01:22:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 -- start start 2024-09-05T01:22:19.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e6540 0x5598b10eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3998012750 con 0x5598b10e14b0 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3998010020 con 0x5598b10e0b70 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.473+0000 7f39b4616740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f399800de70 con 0x5598b10e6540 2024-09-05T01:22:19.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58526/0 (socket says 172.21.15.129:58526) 2024-09-05T01:22:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 -- 172.21.15.129:0/230092708 learned_addr learned my addr 172.21.15.129:0/230092708 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aaffd640 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 -- 172.21.15.129:0/230092708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e6540 msgr2=0x5598b10eb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e6540 0x5598b10eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 -- 172.21.15.129:0/230092708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 msgr2=0x5598b10eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.474+0000 7f39aa7fc640 1 -- 172.21.15.129:0/230092708 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b10ea930 con 0x5598b10e14b0 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.475+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006c70 tx=0x7f39a0007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b5bd1f72528f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.475+0000 7f3990ff9640 1 -- 172.21.15.129:0/230092708 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0023420 con 0x5598b10e14b0 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.476+0000 7f3990ff9640 1 -- 172.21.15.129:0/230092708 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39a001e030 con 0x5598b10e14b0 2024-09-05T01:22:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.476+0000 7f3990ff9640 1 -- 172.21.15.129:0/230092708 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00226c0 con 0x5598b10e14b0 2024-09-05T01:22:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 -- 172.21.15.129:0/230092708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 msgr2=0x5598b10f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f39a0006c70 tx=0x7f39a0007820 comp rx=0 tx=0).stop 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 -- 172.21.15.129:0/230092708 shutdown_connections 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e6540 0x5598b10eb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e14b0 0x5598b10f2290 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 --2- 172.21.15.129:0/230092708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 -- 172.21.15.129:0/230092708 >> 172.21.15.129:0/230092708 conn(0x5598b10dc1e0 msgr2=0x5598b10e26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 -- 172.21.15.129:0/230092708 shutdown_connections 2024-09-05T01:22:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.478+0000 7f39b4616740 1 -- 172.21.15.129:0/230092708 wait complete. 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 -- start start 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 Processor -- start 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f399800e4e0 con 0x5598b10e6540 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f399800f950 con 0x5598b10e0b70 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.479+0000 7f39b4616740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f399800ebb0 con 0x5598b10e14b0 2024-09-05T01:22:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39b4616740 10 client.0 _ll_register_callbacks cb 0x7f39a00071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58534/0 (socket says 172.21.15.129:58534) 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 -- 172.21.15.129:0/16025335 learned_addr learned my addr 172.21.15.129:0/16025335 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39aaffd640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39aa7fc640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 -- 172.21.15.129:0/16025335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 msgr2=0x5598b10ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 -- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 msgr2=0x5598b10eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 -- 172.21.15.129:0/16025335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b10f2470 con 0x5598b10e6540 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f39ab7fe640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f399c0ee210 tx=0x7f399c0bc9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833e227f3b9f47e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.480+0000 7f398eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c049040 con 0x5598b10e6540 2024-09-05T01:22:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f399c0b9640 con 0x5598b10e6540 2024-09-05T01:22:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39a001e030 con 0x5598b10e6540 2024-09-05T01:22:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f399c0bcfd0 con 0x5598b10e6540 2024-09-05T01:22:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f39b4616740 10 client.16044 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.481+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5598b1160830 con 0x5598b10e6540 2024-09-05T01:22:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.483+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f399c0c1380 con 0x5598b10e6540 2024-09-05T01:22:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.483+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f399c0b7b40 con 0x5598b10e6540 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 10 client.16044 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 20 client.16044 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b11611e0 0x5598b1163600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 4 client.16044 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f39a001e030 con 0x5598b11611e0 2024-09-05T01:22:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.484+0000 7f39aaffd640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b11611e0 0x5598b1163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.485+0000 7f39aaffd640 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b11611e0 0x5598b1163600 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8ac1a41536fe49 server_cookie=13060bc397c4c62d in_seq=0 out_seq=0 2024-09-05T01:22:19.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.485+0000 7f398eff5640 10 client.16044 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:19 smithi194 ceph-mon[34941]: pgmap v561: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-09-05T01:22:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.529+0000 7f398eff5640 1 -- 172.21.15.129:0/16025335 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f39a001e030 con 0x5598b11611e0 2024-09-05T01:22:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.529+0000 7f398eff5640 10 client.16044 handle_command_reply: tid=0 2024-09-05T01:22:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.529+0000 7f39b4616740 1 client.16044 shutdown 2024-09-05T01:22:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.529+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 msgr2=0x5598b115f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.529+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f399c0ee210 tx=0x7f399c0bc9f0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 shutdown_connections 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5598b11611e0 0x5598b1163600 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5598b10e6540 0x5598b115f2d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f398eff5640 0 client.16044 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5598b10e14b0 0x5598b10ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 --2- 172.21.15.129:0/16025335 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5598b10e0b70 0x5598b10eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 >> 172.21.15.129:0/16025335 conn(0x5598b10dc1e0 msgr2=0x5598b10daa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.530+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 shutdown_connections 2024-09-05T01:22:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.531+0000 7f39b4616740 1 -- 172.21.15.129:0/16025335 wait complete. 2024-09-05T01:22:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.531+0000 7f39b4616740 20 client.16044 trim_cache size 0 max 16384 2024-09-05T01:22:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:22:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:22:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:22:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:22:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 msgr2=0x7f39940a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 0x7f39940a51a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0518b0 tx=0x7f39ac0505a0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 msgr2=0x7f39a4152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 0x7f39a4152fb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f399c02aa70 tx=0x7f399c003be0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 shutdown_connections 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f39940a2ce0 0x7f39940a51a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f39a40a6030 0x7f39a4152fb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39a40a5670 0x7f39a40b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 --2- 172.21.15.129:0/4208143583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f39a40a4540 0x7f39a40b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.533+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 >> 172.21.15.129:0/4208143583 conn(0x7f39a409fba0 msgr2=0x7f39a40a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.534+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 shutdown_connections 2024-09-05T01:22:19.535 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:19.534+0000 7f39abfff640 1 -- 172.21.15.129:0/4208143583 wait complete. 2024-09-05T01:22:20.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 315.0 s, 959.0 tps, lat 104.429 ms stddev 121.677 2024-09-05T01:22:20.313 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:20 smithi129 ceph-mon[30979]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 993 op/s 2024-09-05T01:22:20.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:20 smithi183 ceph-mon[34589]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 993 op/s 2024-09-05T01:22:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:20 smithi194 ceph-mon[34941]: pgmap v562: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 993 op/s 2024-09-05T01:22:23.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:22 smithi129 ceph-mon[30979]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-05T01:22:23.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:22 smithi194 ceph-mon[34941]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-05T01:22:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:22 smithi183 ceph-mon[34589]: pgmap v563: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-05T01:22:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 320.0 s, 855.6 tps, lat 116.107 ms stddev 135.418 2024-09-05T01:22:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:24 smithi129 ceph-mon[30979]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-09-05T01:22:25.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:24 smithi194 ceph-mon[34941]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-09-05T01:22:25.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:24 smithi183 ceph-mon[34589]: pgmap v564: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 17 MiB/s wr, 1.11k op/s 2024-09-05T01:22:27.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:26 smithi129 ceph-mon[30979]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:22:27.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:26 smithi194 ceph-mon[34941]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:22:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:26 smithi183 ceph-mon[34589]: pgmap v565: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:22:27.645 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T01:22:27.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:22:29.100 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:22:29.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:28 smithi194 ceph-mon[34941]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-09-05T01:22:29.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:28 smithi183 ceph-mon[34589]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-09-05T01:22:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:28 smithi129 ceph-mon[30979]: pgmap v566: 129 pgs: 129 active+clean; 1.3 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 15 MiB/s wr, 1.05k op/s 2024-09-05T01:22:30.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 325.0 s, 982.6 tps, lat 102.542 ms stddev 145.897 2024-09-05T01:22:30.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.299+0000 7f3f719ac640 1 -- 172.21.15.129:0/1943420623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c100e20 msgr2=0x7f3f6c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:30.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.299+0000 7f3f719ac640 1 --2- 172.21.15.129:0/1943420623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c100e20 0x7f3f6c101200 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009930 tx=0x7f3f5c02a4d0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.300+0000 7f3f719ac640 1 -- 172.21.15.129:0/1943420623 shutdown_connections 2024-09-05T01:22:30.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.300+0000 7f3f719ac640 1 --2- 172.21.15.129:0/1943420623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 0x7f3f6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.300+0000 7f3f719ac640 1 --2- 172.21.15.129:0/1943420623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c102570 0x7f3f6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.300+0000 7f3f719ac640 1 --2- 172.21.15.129:0/1943420623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c100e20 0x7f3f6c101200 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.300+0000 7f3f719ac640 1 -- 172.21.15.129:0/1943420623 >> 172.21.15.129:0/1943420623 conn(0x7f3f6c0fc7d0 msgr2=0x7f3f6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:30.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.301+0000 7f3f719ac640 1 -- 172.21.15.129:0/1943420623 shutdown_connections 2024-09-05T01:22:30.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.301+0000 7f3f719ac640 1 -- 172.21.15.129:0/1943420623 wait complete. 2024-09-05T01:22:30.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.302+0000 7f3f719ac640 1 Processor -- start 2024-09-05T01:22:30.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.303+0000 7f3f719ac640 1 -- start start 2024-09-05T01:22:30.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.303+0000 7f3f719ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c100e20 0x7f3f6c196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.303+0000 7f3f719ac640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.303+0000 7f3f719ac640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 0x7f3f6c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f719ac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10fd70 con 0x7f3f6c100e20 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f719ac640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10fbf0 con 0x7f3f6c102570 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6b7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 0x7f3f6c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:33164/0 (socket says 172.21.15.129:33164) 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 -- 172.21.15.129:0/3582662801 learned_addr learned my addr 172.21.15.129:0/3582662801 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6c10fef0 con 0x7f3f6c102ef0 2024-09-05T01:22:30.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6affd640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c100e20 0x7f3f6c196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:30.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 -- 172.21.15.129:0/3582662801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 msgr2=0x7f3f6c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:30.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 0x7f3f6c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 -- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c100e20 msgr2=0x7f3f6c196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:30.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c100e20 0x7f3f6c196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.304+0000 7f3f6a7fc640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6c19b120 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f6a7fc640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ce50 tx=0x7f3f6000e420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60018070 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6c19b410 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f600134d0 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6000a6a0 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.305+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6c1a2cf0 con 0x7f3f6c102570 2024-09-05T01:22:30.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.307+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6c101200 con 0x7f3f6c102570 2024-09-05T01:22:30.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.308+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3f6001b070 con 0x7f3f6c102570 2024-09-05T01:22:30.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.309+0000 7f3f709aa640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 0x7f3f540a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:30.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.309+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f3f600d2e50 con 0x7f3f6c102570 2024-09-05T01:22:30.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.310+0000 7f3f6affd640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 0x7f3f540a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:30.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.311+0000 7f3f6affd640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 0x7f3f540a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006010 tx=0x7f3f5c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:30.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.313+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f60022020 con 0x7f3f6c102570 2024-09-05T01:22:30.711 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.709+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f6c109090 con 0x7f3f6c102570 2024-09-05T01:22:30.713 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:22:30.713 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:22:30.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.711+0000 7f3f709aa640 1 -- 172.21.15.129:0/3582662801 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f3f6009f940 con 0x7f3f6c102570 2024-09-05T01:22:30.714 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:22:30.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 msgr2=0x7f3f540a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:30.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 0x7f3f540a5140 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c006010 tx=0x7f3f5c031400 comp rx=0 tx=0).stop 2024-09-05T01:22:30.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 msgr2=0x7f3f6c1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:30.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000ce50 tx=0x7f3f6000e420 comp rx=0 tx=0).stop 2024-09-05T01:22:30.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 shutdown_connections 2024-09-05T01:22:30.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3f540a2c80 0x7f3f540a5140 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3f6c102ef0 0x7f3f6c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f6c102570 0x7f3f6c1966b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 --2- 172.21.15.129:0/3582662801 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3f6c100e20 0x7f3f6c196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:30.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.717+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 >> 172.21.15.129:0/3582662801 conn(0x7f3f6c0fc7d0 msgr2=0x7f3f6c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:30.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.720+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 shutdown_connections 2024-09-05T01:22:30.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:30.721+0000 7f3f719ac640 1 -- 172.21.15.129:0/3582662801 wait complete. 2024-09-05T01:22:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:30 smithi194 ceph-mon[34941]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-09-05T01:22:31.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3582662801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:31.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:30 smithi183 ceph-mon[34589]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-09-05T01:22:31.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3582662801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:30 smithi129 ceph-mon[30979]: pgmap v567: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 920 op/s 2024-09-05T01:22:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3582662801' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:31.613 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T01:22:31.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T01:22:32.270 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:22:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 -- 172.21.15.129:0/115236408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109380 msgr2=0x7f6498109800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 --2- 172.21.15.129:0/115236408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109380 0x7f6498109800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f648c009900 tx=0x7f648c02a420 comp rx=0 tx=0).stop 2024-09-05T01:22:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 -- 172.21.15.129:0/115236408 shutdown_connections 2024-09-05T01:22:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 --2- 172.21.15.129:0/115236408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109d40 0x7f64981105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 --2- 172.21.15.129:0/115236408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109380 0x7f6498109800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 --2- 172.21.15.129:0/115236408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6498108470 0x7f6498108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.932+0000 7f649d265640 1 -- 172.21.15.129:0/115236408 >> 172.21.15.129:0/115236408 conn(0x7f6498077490 msgr2=0x7f64980798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.934+0000 7f649d265640 1 -- 172.21.15.129:0/115236408 shutdown_connections 2024-09-05T01:22:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.934+0000 7f649d265640 1 -- 172.21.15.129:0/115236408 wait complete. 2024-09-05T01:22:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 Processor -- start 2024-09-05T01:22:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 -- start start 2024-09-05T01:22:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109380 0x7f649819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109d40 0x7f649819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6498113d70 con 0x7f6498109380 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6498113bf0 con 0x7f64981a3980 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f649d265640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6498113ef0 con 0x7f6498109d40 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f64977fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f64977fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35318/0 (socket says 172.21.15.129:35318) 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.935+0000 7f64977fe640 1 -- 172.21.15.129:0/1421218370 learned_addr learned my addr 172.21.15.129:0/1421218370 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64967fc640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109d40 0x7f649819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109380 0x7f649819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 -- 172.21.15.129:0/1421218370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109d40 msgr2=0x7f649819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109d40 0x7f649819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 -- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109380 msgr2=0x7f649819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109380 0x7f649819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 -- 172.21.15.129:0/1421218370 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649806bd40 con 0x7f64981a3980 2024-09-05T01:22:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.936+0000 7f64977fe640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f648800c970 tx=0x7f648800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.937+0000 7f647ffff640 1 -- 172.21.15.129:0/1421218370 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488007bb0 con 0x7f64981a3980 2024-09-05T01:22:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.937+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649806c030 con 0x7f64981a3980 2024-09-05T01:22:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.937+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649806c5d0 con 0x7f64981a3980 2024-09-05T01:22:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.937+0000 7f647ffff640 1 -- 172.21.15.129:0/1421218370 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6488007d50 con 0x7f64981a3980 2024-09-05T01:22:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.940+0000 7f647ffff640 1 -- 172.21.15.129:0/1421218370 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648800f450 con 0x7f64981a3980 2024-09-05T01:22:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.941+0000 7f647ffff640 1 -- 172.21.15.129:0/1421218370 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f648800f6b0 con 0x7f64981a3980 2024-09-05T01:22:32.942 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.941+0000 7f942f1f5640 1 -- 172.21.15.129:0/3825646950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9428105d90 msgr2=0x7f9428106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.941+0000 7f942f1f5640 1 --2- 172.21.15.129:0/3825646950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9428105d90 0x7f9428106220 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9418009900 tx=0x7f941802a490 comp rx=0 tx=0).stop 2024-09-05T01:22:32.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.942+0000 7f942f1f5640 1 -- 172.21.15.129:0/3825646950 shutdown_connections 2024-09-05T01:22:32.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.942+0000 7f942f1f5640 1 --2- 172.21.15.129:0/3825646950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9428105d90 0x7f9428106220 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.942+0000 7f942f1f5640 1 --2- 172.21.15.129:0/3825646950 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94281012a0 0x7f9428105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.942+0000 7f942f1f5640 1 --2- 172.21.15.129:0/3825646950 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.942+0000 7f942f1f5640 1 -- 172.21.15.129:0/3825646950 >> 172.21.15.129:0/3825646950 conn(0x7f94280fc610 msgr2=0x7f94280fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.944 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.943+0000 7f942f1f5640 1 -- 172.21.15.129:0/3825646950 shutdown_connections 2024-09-05T01:22:32.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.943+0000 7f942f1f5640 1 -- 172.21.15.129:0/3825646950 wait complete. 2024-09-05T01:22:32.946 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f942f1f5640 1 Processor -- start 2024-09-05T01:22:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f647ffff640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 0x7f64800a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f647ffff640 1 -- 172.21.15.129:0/1421218370 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f64880d0180 con 0x7f64981a3980 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f942f1f5640 1 -- start start 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f942f1f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f942f1f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 0x7f942806cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.944+0000 7f942f1f5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942f1f5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9428118260 con 0x7f94281012a0 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942f1f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94281180e0 con 0x7f9428100980 2024-09-05T01:22:32.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942f1f5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94281183e0 con 0x7f9428105d90 2024-09-05T01:22:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 0x7f64800a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f64a0001740 1 Processor -- start 2024-09-05T01:22:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f64a0001740 1 -- start start 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942e9f4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942e9f4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48164/0 (socket says 172.21.15.129:48164) 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.945+0000 7f942e9f4640 1 -- 172.21.15.129:0/2891996762 learned_addr learned my addr 172.21.15.129:0/2891996762 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 0x7f942806cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 -- 172.21.15.129:0/2891996762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 msgr2=0x7f942806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 -- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 msgr2=0x7f942806cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 0x7f942806cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 -- 172.21.15.129:0/2891996762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942806d680 con 0x7f9428100980 2024-09-05T01:22:32.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e9f4640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:32.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 0x7f942806cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f64a0001740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b093da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64a0001740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.951 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.946+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428073a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f942000dae0 tx=0x7f942000b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64a0001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b09b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64967fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64967fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48174/0 (socket says 172.21.15.129:48174) 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64967fc640 1 -- 172.21.15.129:0/2462764007 learned_addr learned my addr 172.21.15.129:0/2462764007 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b62aff72f0 con 0x55b62b099370 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62af9f270 con 0x55b62b0939a0 2024-09-05T01:22:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62aff32b0 con 0x55b62b0942e0 2024-09-05T01:22:32.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f941f7fe640 1 -- 172.21.15.129:0/2891996762 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420017030 con 0x7f9428100980 2024-09-05T01:22:32.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f941f7fe640 1 -- 172.21.15.129:0/2891996762 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9420003e20 con 0x7f9428100980 2024-09-05T01:22:32.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94280715a0 con 0x7f9428100980 2024-09-05T01:22:32.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f941f7fe640 1 -- 172.21.15.129:0/2891996762 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420010460 con 0x7f9428100980 2024-09-05T01:22:32.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9428071b30 con 0x7f9428100980 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.947+0000 7f6496ffd640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b093da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64977fe640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b09b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 -- 172.21.15.129:0/2462764007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 msgr2=0x55b62b093da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b093da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 -- 172.21.15.129:0/2462764007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 msgr2=0x55b62b09b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b09b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 -- 172.21.15.129:0/2462764007 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62af9e340 con 0x55b62b0942e0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64977fe640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b09b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f64967fc640 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f648c009900 tx=0x7f648c004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b79f2c9f8a8cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.948+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 0x7f64800a5310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6490005c40 tx=0x7f649000a2b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.949+0000 7f647d7fa640 1 -- 172.21.15.129:0/2462764007 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c035030 con 0x55b62b0942e0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.949+0000 7f647d7fa640 1 -- 172.21.15.129:0/2462764007 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f648c003aa0 con 0x55b62b0942e0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.949+0000 7f647d7fa640 1 -- 172.21.15.129:0/2462764007 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c031cb0 con 0x55b62b0942e0 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 msgr2=0x55b62b098da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f648c009900 tx=0x7f648c004200 comp rx=0 tx=0).stop 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 shutdown_connections 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b09b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b098da0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 --2- 172.21.15.129:0/2462764007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b093da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.950+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 >> 172.21.15.129:0/2462764007 conn(0x55b62b08f010 msgr2=0x55b62b08f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.951+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 shutdown_connections 2024-09-05T01:22:32.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.951+0000 7f941f7fe640 1 -- 172.21.15.129:0/2891996762 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f942001a070 con 0x7f9428100980 2024-09-05T01:22:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.952+0000 7f64a0001740 1 -- 172.21.15.129:0/2462764007 wait complete. 2024-09-05T01:22:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.952+0000 7f64a0001740 1 -- start start 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.952+0000 7f64a0001740 1 Processor -- start 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b09dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b09e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64967fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b09e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b62b09c2e0 con 0x55b62b099370 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b62af80290 con 0x55b62b0939a0 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64967fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b09e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48184/0 (socket says 172.21.15.129:48184) 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64967fc640 1 -- 172.21.15.129:0/3672903155 learned_addr learned my addr 172.21.15.129:0/3672903155 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b62b09c460 con 0x55b62b0942e0 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f64a0001740 10 client.0 _ll_register_callbacks cb 0x55b62afb1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b09dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 -- 172.21.15.129:0/3672903155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 msgr2=0x55b62b09e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b09e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 -- 172.21.15.129:0/3672903155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 msgr2=0x55b62b0a51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f6496ffd640 1 -- 172.21.15.129:0/3672903155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62ad0a210 con 0x55b62b0939a0 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f6496ffd640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b09dc50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f649807a880 tx=0x7f6490004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c874bb792db07a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f647aff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f649001db50 con 0x55b62b0939a0 2024-09-05T01:22:32.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f649001dcf0 con 0x55b62b0939a0 2024-09-05T01:22:32.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.952+0000 7f941f7fe640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 0x7f94140a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 0x7f94140a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f9432830740 1 Processor -- start 2024-09-05T01:22:32.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f941f7fe640 1 -- 172.21.15.129:0/2891996762 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f94200ce4a0 con 0x7f9428100980 2024-09-05T01:22:32.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.953+0000 7f9432830740 1 -- start start 2024-09-05T01:22:32.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f648c003aa0 con 0x55b62b0939a0 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b0694915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b069493ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b0693ec2a0 con 0x55b069487220 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b069393b10 con 0x55b069486900 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f9432830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b0693e8260 con 0x55b069491ae0 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 0x7f94140a5180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f942806e2c0 tx=0x7f9424005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f942e1f3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f942e1f3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35346/0 (socket says 172.21.15.129:35346) 2024-09-05T01:22:32.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f942e1f3640 1 -- 172.21.15.129:0/2018961114 learned_addr learned my addr 172.21.15.129:0/2018961114 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.954+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b0694915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e9f4640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b069493ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 -- 172.21.15.129:0/2018961114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 msgr2=0x55b069493ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b069493ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 -- 172.21.15.129:0/2018961114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 msgr2=0x55b0694915a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b0694915a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 -- 172.21.15.129:0/2018961114 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b06915db90 con 0x55b069486900 2024-09-05T01:22:32.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b0694915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:32.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f649000ae10 con 0x55b62b0939a0 2024-09-05T01:22:32.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f94200a2fd0 tx=0x7f942009a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7355233867a3d661 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f64a0001740 10 client.25607 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.955+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b62add6620 con 0x55b62b0939a0 2024-09-05T01:22:32.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.956+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6490017070 con 0x55b62b0939a0 2024-09-05T01:22:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f649001e520 con 0x55b62b0939a0 2024-09-05T01:22:32.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f64a0001740 10 client.25607 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:32.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.956+0000 7f941cdea640 1 -- 172.21.15.129:0/2018961114 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200b3070 con 0x55b069486900 2024-09-05T01:22:32.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.956+0000 7f941cdea640 1 -- 172.21.15.129:0/2018961114 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f942009ecd0 con 0x55b069486900 2024-09-05T01:22:32.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f9432830740 1 -- 172.21.15.129:0/2018961114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 msgr2=0x55b069486ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f9432830740 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f94200a2fd0 tx=0x7f942009a560 comp rx=0 tx=0).stop 2024-09-05T01:22:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f64a0001740 10 client.25607 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f64a0001740 20 client.25607 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b0a99e0 0x55b62b0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f9432830740 1 -- 172.21.15.129:0/2018961114 shutdown_connections 2024-09-05T01:22:32.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f9432830740 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b069493ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.957+0000 7f9432830740 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b0694915a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f9432830740 1 --2- 172.21.15.129:0/2018961114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b069486ce0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f9432830740 1 -- 172.21.15.129:0/2018961114 >> 172.21.15.129:0/2018961114 conn(0x55b069482e90 msgr2=0x55b069483270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f64a0001740 4 client.25607 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f648c003aa0 con 0x55b62b0a99e0 2024-09-05T01:22:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f64967fc640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b0a99e0 0x55b62b0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.958+0000 7f9432830740 1 -- 172.21.15.129:0/2018961114 shutdown_connections 2024-09-05T01:22:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.959+0000 7f64967fc640 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b0a99e0 0x55b62b0abdf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f17e50037178b55c server_cookie=a7837e4735a38e8a in_seq=0 out_seq=0 2024-09-05T01:22:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.959+0000 7f647aff5640 10 client.25607 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:32.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.959+0000 7f9432830740 1 -- 172.21.15.129:0/2018961114 wait complete. 2024-09-05T01:22:32.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.959+0000 7f9432830740 1 -- start start 2024-09-05T01:22:32.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.959+0000 7f9432830740 1 Processor -- start 2024-09-05T01:22:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f647aff5640 1 -- 172.21.15.129:0/3672903155 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f648c003aa0 con 0x55b62b0a99e0 2024-09-05T01:22:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f647aff5640 10 client.25607 handle_command_reply: tid=0 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f9432830740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069486900 0x55b06948ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f9432830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069487220 0x55b06948ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f9432830740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.960+0000 7f9432830740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b069494700 con 0x55b069486900 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f9432830740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b0693746e0 con 0x55b069491ae0 2024-09-05T01:22:32.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f9432830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b069494880 con 0x55b069487220 2024-09-05T01:22:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f64a0001740 1 client.25607 shutdown 2024-09-05T01:22:32.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35354/0 (socket says 172.21.15.129:35354) 2024-09-05T01:22:32.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 -- 172.21.15.129:0/3145502413 learned_addr learned my addr 172.21.15.129:0/3145502413 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942d9f2640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069487220 0x55b06948ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 msgr2=0x55b62b09dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b09dc50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f649807a880 tx=0x7f6490004a40 comp rx=0 tx=0).stop 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 -- 172.21.15.129:0/3145502413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069487220 msgr2=0x55b06948ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069487220 0x55b06948ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 -- 172.21.15.129:0/3145502413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069486900 msgr2=0x55b06948ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069486900 0x55b06948ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f942e9f4640 1 -- 172.21.15.129:0/3145502413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b06949dcf0 con 0x55b069491ae0 2024-09-05T01:22:32.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f942e9f4640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9418031440 tx=0x7f9418005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cb891d11ffa754d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.974 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f9411ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:22:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 shutdown_connections 2024-09-05T01:22:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b0a99e0 0x55b62b0abdf0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b09e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 --2- 172.21.15.129:0/3672903155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b09dc50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 >> 172.21.15.129:0/3672903155 conn(0x55b62b08f010 msgr2=0x55b62b090820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f647aff5640 0 client.25607 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 shutdown_connections 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f9411ffb640 1 -- 172.21.15.129:0/3145502413 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418035040 con 0x55b069491ae0 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f9411ffb640 1 -- 172.21.15.129:0/3145502413 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f94180046c0 con 0x55b069491ae0 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.961+0000 7f9432830740 10 client.0 _ll_register_callbacks cb 0x55b069398a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.962+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f942009ecd0 con 0x55b069491ae0 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f9411ffb640 1 -- 172.21.15.129:0/3145502413 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94180048c0 con 0x55b069491ae0 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f9432830740 10 client.25613 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:32.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55b0694a0580 con 0x55b069491ae0 2024-09-05T01:22:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f64a0001740 1 -- 172.21.15.129:0/3672903155 wait complete. 2024-09-05T01:22:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f64a0001740 20 client.25607 trim_cache size 0 max 16384 2024-09-05T01:22:32.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f9411ffb640 1 -- 172.21.15.129:0/3145502413 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f9418004a60 con 0x55b069491ae0 2024-09-05T01:22:32.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.964+0000 7f9432830740 10 client.25613 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:32.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.965+0000 7f9432830740 10 client.25613 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:22:32.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.965+0000 7f9432830740 20 client.25613 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:32.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.965+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b0694a1980 0x55b0694a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.965+0000 7f9432830740 4 client.25613 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.965+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f942009ecd0 con 0x55b0694a1980 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.966+0000 7f942e1f3640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b0694a1980 0x55b0694a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.967+0000 7f942e1f3640 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b0694a1980 0x55b0694a3e40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d28dae355d3166b server_cookie=7d80707c00b1c0d2 in_seq=0 out_seq=0 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.967+0000 7f9411ffb640 10 client.25613 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.969+0000 7f9411ffb640 1 -- 172.21.15.129:0/3145502413 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f942009ecd0 con 0x55b0694a1980 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.969+0000 7f9411ffb640 10 client.25613 handle_command_reply: tid=0 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.969+0000 7f9432830740 1 client.25613 shutdown 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.969+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 msgr2=0x55b06948f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:32.979 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.969+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9418031440 tx=0x7f9418005270 comp rx=0 tx=0).stop 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 shutdown_connections 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b0694a1980 0x55b0694a3e40 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9411ffb640 0 client.25613 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b06948f4a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069487220 0x55b06948ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 --2- 172.21.15.129:0/3145502413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069486900 0x55b06948ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 >> 172.21.15.129:0/3145502413 conn(0x55b069482e90 msgr2=0x55b069488bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 shutdown_connections 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 1 -- 172.21.15.129:0/3145502413 wait complete. 2024-09-05T01:22:32.980 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:32.970+0000 7f9432830740 20 client.25613 trim_cache size 0 max 16384 2024-09-05T01:22:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.994+0000 7f64a0001740 1 Processor -- start 2024-09-05T01:22:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.994+0000 7f64a0001740 1 -- start start 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b08e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6484012630 con 0x55b62b099370 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f648400ff00 con 0x55b62b0939a0 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64a0001740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f648400dd50 con 0x55b62b0942e0 2024-09-05T01:22:32.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64977fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56714/0 (socket says 172.21.15.129:56714) 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.995+0000 7f64977fe640 1 -- 172.21.15.129:0/3346022411 learned_addr learned my addr 172.21.15.129:0/3346022411 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 -- 172.21.15.129:0/3346022411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 msgr2=0x55b62b0a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 -- 172.21.15.129:0/3346022411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 msgr2=0x55b62b08e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b08e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 -- 172.21.15.129:0/3346022411 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62b09d790 con 0x55b62b099370 2024-09-05T01:22:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.996+0000 7f64977fe640 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f648809c580 tx=0x7f648809bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64160bc0368e212 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.997+0000 7f647cff9640 1 -- 172.21.15.129:0/3346022411 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64880b5070 con 0x55b62b099370 2024-09-05T01:22:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.997+0000 7f647cff9640 1 -- 172.21.15.129:0/3346022411 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64880a09a0 con 0x55b62b099370 2024-09-05T01:22:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.997+0000 7f647cff9640 1 -- 172.21.15.129:0/3346022411 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648801d040 con 0x55b62b099370 2024-09-05T01:22:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.998+0000 7f64a0001740 1 -- 172.21.15.129:0/3346022411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 msgr2=0x55b62b090400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.998+0000 7f64a0001740 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f648809c580 tx=0x7f648809bf50 comp rx=0 tx=0).stop 2024-09-05T01:22:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 -- 172.21.15.129:0/3346022411 shutdown_connections 2024-09-05T01:22:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b090400 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0942e0 0x55b62b0a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 --2- 172.21.15.129:0/3346022411 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0939a0 0x55b62b08e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 -- 172.21.15.129:0/3346022411 >> 172.21.15.129:0/3346022411 conn(0x55b62b08f010 msgr2=0x55b62b08d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 -- 172.21.15.129:0/3346022411 shutdown_connections 2024-09-05T01:22:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:32.999+0000 7f64a0001740 1 -- 172.21.15.129:0/3346022411 wait complete. 2024-09-05T01:22:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64a0001740 1 -- start start 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64a0001740 1 Processor -- start 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64a0001740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64a0001740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64a0001740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f6496ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48216/0 (socket says 172.21.15.129:48216) 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64967fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.000+0000 7f64967fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35360/0 (socket says 172.21.15.129:35360) 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 -- 172.21.15.129:0/1339596762 learned_addr learned my addr 172.21.15.129:0/1339596762 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f648400e3c0 con 0x55b62b099370 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f648400f830 con 0x55b62b0942e0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f648400ea90 con 0x55b62b0939a0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64977fe640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 -- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 msgr2=0x55b62b0a2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 -- 172.21.15.129:0/1339596762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 msgr2=0x55b62b0a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62b0ac470 con 0x55b62b0939a0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64967fc640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f64a0001740 10 client.0 _ll_register_callbacks cb 0x7f6490015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.001+0000 7f6496ffd640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649001e6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb40b8ba3f4588a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64880a09a0 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f647aff5640 10 client.25564 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f647aff5640 1 -- 172.21.15.129:0/1339596762 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490004d40 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f647aff5640 1 -- 172.21.15.129:0/1339596762 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f649001d9b0 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f647aff5640 1 -- 172.21.15.129:0/1339596762 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f649001dc30 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f64a0001740 10 client.25564 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.002+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55b62b0a4f00 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f647aff5640 1 -- 172.21.15.129:0/1339596762 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6490031470 con 0x55b62b0939a0 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 10 client.25564 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 10 client.25564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 20 client.25564 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b1147b0 0x55b62b116c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 4 client.25564 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:22:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b62b004a20 con 0x55b62b1147b0 2024-09-05T01:22:33.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.003+0000 7f9432830740 1 Processor -- start 2024-09-05T01:22:33.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 -- start start 2024-09-05T01:22:33.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b0694834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b069488ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b0694a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940c0126e0 con 0x55b069487220 2024-09-05T01:22:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f940c00ffb0 con 0x55b069486900 2024-09-05T01:22:33.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f9432830740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f940c00de00 con 0x55b069491ae0 2024-09-05T01:22:33.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f64967fc640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b1147b0 0x55b62b116c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f942d9f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b069488ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f942e1f3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b0694834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f942d9f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b069488ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56740/0 (socket says 172.21.15.129:56740) 2024-09-05T01:22:33.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f942d9f2640 1 -- 172.21.15.129:0/1115939421 learned_addr learned my addr 172.21.15.129:0/1115939421 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:33.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.004+0000 7f942e9f4640 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b0694a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f64967fc640 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b1147b0 0x55b62b116c70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8becc1361950d6ae server_cookie=26b644b2f5d81fc0 in_seq=0 out_seq=0 2024-09-05T01:22:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f647aff5640 10 client.25564 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 -- 172.21.15.129:0/1115939421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 msgr2=0x55b0694a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b0694a9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 -- 172.21.15.129:0/1115939421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 msgr2=0x55b069488ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b069488ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 -- 172.21.15.129:0/1115939421 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b069486710 con 0x55b069486900 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f942e1f3640 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b0694834a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94200a2fd0 tx=0x7f9420004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62195f50d71d6ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:33.017 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f9413fff640 1 -- 172.21.15.129:0/1115939421 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94200b3050 con 0x55b069486900 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f9413fff640 1 -- 172.21.15.129:0/1115939421 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9420098c20 con 0x55b069486900 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.005+0000 7f9413fff640 1 -- 172.21.15.129:0/1115939421 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000bc40 con 0x55b069486900 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 -- 172.21.15.129:0/1115939421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 msgr2=0x55b0694834a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b0694834a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94200a2fd0 tx=0x7f9420004990 comp rx=0 tx=0).stop 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 -- 172.21.15.129:0/1115939421 shutdown_connections 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b069491ae0 0x55b0694a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b069488ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 --2- 172.21.15.129:0/1115939421 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069486900 0x55b0694834a0 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f94200a2fd0 tx=0x7f9420004990 comp rx=0 tx=0).stop 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.006+0000 7f9432830740 1 -- 172.21.15.129:0/1115939421 >> 172.21.15.129:0/1115939421 conn(0x55b069482e90 msgr2=0x55b0694883a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.007+0000 7f9432830740 1 -- 172.21.15.129:0/1115939421 shutdown_connections 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.007+0000 7f9432830740 1 -- 172.21.15.129:0/1115939421 wait complete. 2024-09-05T01:22:33.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.007+0000 7f9432830740 1 -- start start 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.007+0000 7f9432830740 1 Processor -- start 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f9432830740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f9432830740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942e1f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942e1f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56754/0 (socket says 172.21.15.129:56754) 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f9432830740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0694a1980 0x55b069496a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942d9f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942d9f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35372/0 (socket says 172.21.15.129:35372) 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942e1f3640 1 -- 172.21.15.129:0/2622532585 learned_addr learned my addr 172.21.15.129:0/2622532585 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f940c00e470 con 0x55b069487220 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f940c00f8e0 con 0x55b069491ae0 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.008+0000 7f942e9f4640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0694a1980 0x55b069496a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.019 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f940c00eb40 con 0x55b0694a1980 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f9432830740 10 client.0 _ll_register_callbacks cb 0x7f9418004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 -- 172.21.15.129:0/2622532585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0694a1980 msgr2=0x55b069496a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0694a1980 0x55b069496a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 -- 172.21.15.129:0/2622532585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 msgr2=0x55b06949a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b069497230 con 0x55b069491ae0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f942d9f2640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f942806e350 tx=0x7f9424004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58a7697f4b96b0e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.009+0000 7f9411ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424030060 con 0x55b069491ae0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9420098c20 con 0x55b069491ae0 2024-09-05T01:22:33.020 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f9424012650 con 0x55b069491ae0 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9424037050 con 0x55b069491ae0 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9432830740 10 client.25622 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.010+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b06950d700 con 0x55b069491ae0 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.011+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9420098c20 con 0x55b069491ae0 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.011+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f942402a030 con 0x55b069491ae0 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 10 client.25622 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 10 client.25622 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 20 client.25622 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b06950df00 0x55b0695102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 4 client.25622 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55b0693f99d0 con 0x55b06950df00 2024-09-05T01:22:33.021 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.012+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b06950df00 0x55b0695102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:33.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.016+0000 7f942e1f3640 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b06950df00 0x55b0695102f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46bf488455a03c0 server_cookie=cc05203e98feaf54 in_seq=0 out_seq=0 2024-09-05T01:22:33.022 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.017+0000 7f9411ffb640 10 client.25622 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f647aff5640 1 -- 172.21.15.129:0/1339596762 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b62b004a20 con 0x55b62b1147b0 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f647aff5640 10 client.25564 handle_command_reply: tid=0 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 client.25564 shutdown 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 msgr2=0x55b62b0a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6490006010 tx=0x7f649001e6f0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 shutdown_connections 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b62b1147b0 0x55b62b116c70 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b62b099370 0x55b62b0a3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b62b0942e0 0x55b62b0a2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 --2- 172.21.15.129:0/1339596762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b62b0939a0 0x55b62b0a25b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f647aff5640 0 client.25564 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 >> 172.21.15.129:0/1339596762 conn(0x55b62b08f010 msgr2=0x55b62b08e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 shutdown_connections 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.215+0000 7f64a0001740 1 -- 172.21.15.129:0/1339596762 wait complete. 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f64a0001740 20 client.25564 trim_cache size 0 max 16384 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:22:33.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:22:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 msgr2=0x7f64800a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 0x7f64800a5310 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6490005c40 tx=0x7f649000a2b0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 msgr2=0x7f64981a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f648800c970 tx=0x7f648800ce30 comp rx=0 tx=0).stop 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 shutdown_connections 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f64800a2e50 0x7f64800a5310 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f64981a3980 0x7f64981a5d70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6498109d40 0x7f649819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 --2- 172.21.15.129:0/1421218370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6498109380 0x7f649819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 >> 172.21.15.129:0/1421218370 conn(0x7f6498077490 msgr2=0x7f649810c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 shutdown_connections 2024-09-05T01:22:33.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:33.216+0000 7f649d265640 1 -- 172.21.15.129:0/1421218370 wait complete. 2024-09-05T01:22:33.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:32 smithi129 ceph-mon[30979]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 847 op/s 2024-09-05T01:22:33.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:32 smithi194 ceph-mon[34941]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 847 op/s 2024-09-05T01:22:33.329 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9411ffb640 1 -- 172.21.15.129:0/2622532585 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b0693f99d0 con 0x55b06950df00 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9411ffb640 10 client.25622 handle_command_reply: tid=0 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 client.25622 shutdown 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 msgr2=0x55b0694964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f942806e350 tx=0x7f9424004730 comp rx=0 tx=0).stop 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 shutdown_connections 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b06950df00 0x55b0695102f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b0694a1980 0x55b069496a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9411ffb640 0 client.25622 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b069491ae0 0x55b0694964c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 --2- 172.21.15.129:0/2622532585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b069487220 0x55b06949a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.327+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 >> 172.21.15.129:0/2622532585 conn(0x55b069482e90 msgr2=0x55b069482000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 shutdown_connections 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f9432830740 1 -- 172.21.15.129:0/2622532585 wait complete. 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f9432830740 20 client.25622 trim_cache size 0 max 16384 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 22, 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:22:33.331 INFO:teuthology.orchestra.run.smithi129.stdout: "fcd4421c": { 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 22, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:22:33.332 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:33.333 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 msgr2=0x7f94140a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 0x7f94140a5180 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f942806e2c0 tx=0x7f9424005f90 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 msgr2=0x7f9428073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428073a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f942000dae0 tx=0x7f942000b260 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 shutdown_connections 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94140a2cc0 0x7f94140a5180 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428105d90 0x7f942806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94281012a0 0x7f942806cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 --2- 172.21.15.129:0/2891996762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428100980 0x7f9428073a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:33.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 >> 172.21.15.129:0/2891996762 conn(0x7f94280fc610 msgr2=0x7f942810f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:33.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 shutdown_connections 2024-09-05T01:22:33.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:33.328+0000 7f942f1f5640 1 -- 172.21.15.129:0/2891996762 wait complete. 2024-09-05T01:22:33.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:32 smithi183 ceph-mon[34589]: pgmap v568: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 12 MiB/s wr, 847 op/s 2024-09-05T01:22:33.995 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fcd4421c', quiesce took 6.3 seconds. Will release after: 3.7 2024-09-05T01:22:35.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 330.0 s, 523.4 tps, lat 107.216 ms stddev 62.516 2024-09-05T01:22:35.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:35 smithi183 ceph-mon[34589]: pgmap v569: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-05T01:22:35.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:35 smithi129 ceph-mon[30979]: pgmap v569: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-05T01:22:35.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:35 smithi194 ceph-mon[34941]: pgmap v569: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-05T01:22:37.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:37 smithi183 ceph-mon[34589]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-05T01:22:37.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:37 smithi129 ceph-mon[30979]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-05T01:22:37.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:37 smithi194 ceph-mon[34941]: pgmap v570: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-05T01:22:37.695 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:22:37.955 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:22:38.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 -- 172.21.15.129:0/2194824232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 msgr2=0x7f53901067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:38.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 --2- 172.21.15.129:0/2194824232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901067f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5384009900 tx=0x7f538402a410 comp rx=0 tx=0).stop 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 -- 172.21.15.129:0/2194824232 shutdown_connections 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 --2- 172.21.15.129:0/2194824232 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390106f20 0x7f539010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 --2- 172.21.15.129:0/2194824232 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901067f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 --2- 172.21.15.129:0/2194824232 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390101df0 0x7f53901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 -- 172.21.15.129:0/2194824232 >> 172.21.15.129:0/2194824232 conn(0x7f53900fd8f0 msgr2=0x7f53900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 -- 172.21.15.129:0/2194824232 shutdown_connections 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.450+0000 7f5397a9f640 1 -- 172.21.15.129:0/2194824232 wait complete. 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 Processor -- start 2024-09-05T01:22:38.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 -- start start 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390101df0 0x7f5390198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390106f20 0x7f539019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111e30 con 0x7f53901027a0 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111cb0 con 0x7f5390101df0 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f5397a9f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5390111fb0 con 0x7f5390106f20 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56776/0 (socket says 172.21.15.129:56776) 2024-09-05T01:22:38.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 -- 172.21.15.129:0/3013675155 learned_addr learned my addr 172.21.15.129:0/3013675155 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539583b640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390101df0 0x7f5390198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539603c640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390106f20 0x7f539019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 -- 172.21.15.129:0/3013675155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390106f20 msgr2=0x7f539019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390106f20 0x7f539019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 -- 172.21.15.129:0/3013675155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390101df0 msgr2=0x7f5390198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390101df0 0x7f5390198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.451+0000 7f539503a640 1 -- 172.21.15.129:0/3013675155 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539019d330 con 0x7f53901027a0 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f539503a640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5384005940 tx=0x7f53840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538403f030 con 0x7f53901027a0 2024-09-05T01:22:38.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539019d5c0 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5384003cb0 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384035070 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.452+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53901a4f00 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.453+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53901032d0 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.454+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5384042030 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.455+0000 7f538effd640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 0x7f53780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.455+0000 7f539583b640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 0x7f53780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.455+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f53840ef8a0 con 0x7f53901027a0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.455+0000 7f539583b640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 0x7f53780a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5388005fd0 tx=0x7f5388009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:38.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.459+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53840bc3f0 con 0x7f53901027a0 2024-09-05T01:22:38.658 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.657+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53901996e0 con 0x7f53901027a0 2024-09-05T01:22:38.659 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.658+0000 7f538effd640 1 -- 172.21.15.129:0/3013675155 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f53840bbcf0 con 0x7f53901027a0 2024-09-05T01:22:38.659 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:22:38.660 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:22:38.660 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:22:38.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 msgr2=0x7f53780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 0x7f53780a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5388005fd0 tx=0x7f5388009500 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 msgr2=0x7f53901988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5384005940 tx=0x7f53840042f0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 shutdown_connections 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f53780a2b80 0x7f53780a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5390106f20 0x7f539019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f53901027a0 0x7f53901988c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 --2- 172.21.15.129:0/3013675155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5390101df0 0x7f5390198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 >> 172.21.15.129:0/3013675155 conn(0x7f53900fd8f0 msgr2=0x7f53900fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 shutdown_connections 2024-09-05T01:22:38.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:38.661+0000 7f5397a9f640 1 -- 172.21.15.129:0/3013675155 wait complete. 2024-09-05T01:22:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:39 smithi129 ceph-mon[30979]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-05T01:22:39.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3013675155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:39.397 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id fcd4421c --release --await' 2024-09-05T01:22:39.398 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id fcd4421c --release --await 2024-09-05T01:22:39.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:39 smithi183 ceph-mon[34589]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-05T01:22:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:39 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3013675155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:39.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:39 smithi194 ceph-mon[34941]: pgmap v571: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-05T01:22:39.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:39 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3013675155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:22:39.645 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 -- 172.21.15.129:0/3223309288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178101df0 msgr2=0x7ff1781021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 --2- 172.21.15.129:0/3223309288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178101df0 0x7ff1781021d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff168009ca0 tx=0x7ff16802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 -- 172.21.15.129:0/3223309288 shutdown_connections 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 --2- 172.21.15.129:0/3223309288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff178106f20 0x7ff17810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 --2- 172.21.15.129:0/3223309288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1781027a0 0x7ff1781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 --2- 172.21.15.129:0/3223309288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178101df0 0x7ff1781021d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 -- 172.21.15.129:0/3223309288 >> 172.21.15.129:0/3223309288 conn(0x7ff1780fd8f0 msgr2=0x7ff1780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 -- 172.21.15.129:0/3223309288 shutdown_connections 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.188+0000 7ff180651640 1 -- 172.21.15.129:0/3223309288 wait complete. 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 Processor -- start 2024-09-05T01:22:40.200 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 -- start start 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 0x7ff1780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 0x7ff178074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff178111ed0 con 0x7ff178106f20 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff178111d50 con 0x7ff178101df0 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff180651640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff178112050 con 0x7ff1781027a0 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56800/0 (socket says 172.21.15.129:56800) 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 -- 172.21.15.129:0/3874588562 learned_addr learned my addr 172.21.15.129:0/3874588562 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 0x7ff178074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 0x7ff1780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.201 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 -- 172.21.15.129:0/3874588562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 msgr2=0x7ff178074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 0x7ff178074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 -- 172.21.15.129:0/3874588562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 msgr2=0x7ff1780744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 0x7ff1780744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17ebee640 1 -- 172.21.15.129:0/3874588562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1781a68f0 con 0x7ff178106f20 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 0x7ff178074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.189+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 0x7ff1780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff17ebee640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff170009fd0 tx=0x7ff17000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff1777fe640 1 -- 172.21.15.129:0/3874588562 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170011830 con 0x7ff178106f20 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff1777fe640 1 -- 172.21.15.129:0/3874588562 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff170016070 con 0x7ff178106f20 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1781a6be0 con 0x7ff178106f20 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff1777fe640 1 -- 172.21.15.129:0/3874588562 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170010de0 con 0x7ff178106f20 2024-09-05T01:22:40.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.190+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1781a7120 con 0x7ff178106f20 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.192+0000 7ff1777fe640 1 -- 172.21.15.129:0/3874588562 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff1700119d0 con 0x7ff178106f20 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.192+0000 7ff182a2a740 1 Processor -- start 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 -- start start 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486803440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486810630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560486768620 con 0x560486803980 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56048670fe50 con 0x56048680e240 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff182a2a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5604867645e0 con 0x560486803060 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56814/0 (socket says 172.21.15.129:56814) 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 -- 172.21.15.129:0/658323230 learned_addr learned my addr 172.21.15.129:0/658323230 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff1777fe640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 0x7ff1600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17dbec640 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486803440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17dbec640 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486810630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 -- 172.21.15.129:0/658323230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 msgr2=0x560486803440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486803440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 -- 172.21.15.129:0/658323230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 msgr2=0x560486810630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17ebee640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 0x7ff1600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486810630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 -- 172.21.15.129:0/658323230 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560486700d40 con 0x560486803980 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff1777fe640 1 -- 172.21.15.129:0/3874588562 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff1700ceba0 con 0x7ff178106f20 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.193+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff168035000 tx=0x7ff168009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fd264a4fc6a146 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff174ff9640 1 -- 172.21.15.129:0/658323230 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168031450 con 0x560486803980 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff174ff9640 1 -- 172.21.15.129:0/658323230 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff174ff9640 1 -- 172.21.15.129:0/658323230 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168004440 con 0x560486803980 2024-09-05T01:22:40.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff17ebee640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 0x7ff1600a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff1700979c0 tx=0x7ff1700bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 -- 172.21.15.129:0/658323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 msgr2=0x56048680dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff168035000 tx=0x7ff168009a20 comp rx=0 tx=0).stop 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 -- 172.21.15.129:0/658323230 shutdown_connections 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486810630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680dd00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 --2- 172.21.15.129:0/658323230 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486803440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.194+0000 7ff182a2a740 1 -- 172.21.15.129:0/658323230 >> 172.21.15.129:0/658323230 conn(0x5604867ff5f0 msgr2=0x5604867ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- 172.21.15.129:0/658323230 shutdown_connections 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- 172.21.15.129:0/658323230 wait complete. 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- start start 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 Processor -- start 2024-09-05T01:22:40.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x56048680b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x56048680bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560486810e10 con 0x560486803980 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5604866f0f10 con 0x560486803060 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560486810f90 con 0x56048680e240 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff182a2a740 10 client.0 _ll_register_callbacks cb 0x560486714db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56818/0 (socket says 172.21.15.129:56818) 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3233727294 learned_addr learned my addr 172.21.15.129:0/3233727294 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.195+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x56048680bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x56048680b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3233727294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 msgr2=0x56048680bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x56048680bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 msgr2=0x56048680b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x56048680b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3233727294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560486477580 con 0x560486803980 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x56048680bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff168009620 tx=0x7ff168031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2baf505b56f5833 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168052020 con 0x560486803980 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff168003b70 con 0x560486803980 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff16805d050 con 0x560486803980 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 10 client.16092 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560486541960 con 0x560486803980 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff16804d030 con 0x560486803980 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 10 client.16092 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 10 client.16092 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 20 client.16092 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.196+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56048681dc00 0x56048681fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.197+0000 7ff182a2a740 4 client.16092 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.197+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560486775b90 con 0x56048681dc00 2024-09-05T01:22:40.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.197+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56048681dc00 0x56048681fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.197+0000 7ff15dffb640 10 client.16092 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.197+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56048681dc00 0x56048681fff0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf188365688a5f7 server_cookie=c0956cf1a1f46857 in_seq=0 out_seq=0 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff15dffb640 1 -- 172.21.15.129:0/3233727294 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560486775b90 con 0x56048681dc00 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff15dffb640 10 client.16092 handle_command_reply: tid=0 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 client.16092 shutdown 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 msgr2=0x56048680b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff168009620 tx=0x7ff168031560 comp rx=0 tx=0).stop 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 shutdown_connections 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56048681dc00 0x56048681fff0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x56048680bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x56048680b610 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3233727294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x56048680b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 >> 172.21.15.129:0/3233727294 conn(0x5604867ff5f0 msgr2=0x5604868052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff15dffb640 0 client.16092 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 shutdown_connections 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 1 -- 172.21.15.129:0/3233727294 wait complete. 2024-09-05T01:22:40.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.198+0000 7ff182a2a740 20 client.16092 trim_cache size 0 max 16384 2024-09-05T01:22:40.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 335.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 Processor -- start 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 -- start start 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486805ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486819920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781b3950 con 0x560486803980 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781b1220 con 0x56048680e240 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff182a2a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781af070 con 0x560486803060 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56830/0 (socket says 172.21.15.129:56830) 2024-09-05T01:22:40.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.215+0000 7ff17e3ed640 1 -- 172.21.15.129:0/1683751270 learned_addr learned my addr 172.21.15.129:0/1683751270 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17dbec640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486805ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17dbec640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486819920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 -- 172.21.15.129:0/1683751270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 msgr2=0x560486805ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486805ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 -- 172.21.15.129:0/1683751270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 msgr2=0x560486819920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486819920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 -- 172.21.15.129:0/1683751270 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56048680d010 con 0x560486803980 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17dbec640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486819920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff168000f80 tx=0x7ff168009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29422ee0cb22718 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff15ffff640 1 -- 172.21.15.129:0/1683751270 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168052040 con 0x560486803980 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff15ffff640 1 -- 172.21.15.129:0/1683751270 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.216+0000 7ff15ffff640 1 -- 172.21.15.129:0/1683751270 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16804a040 con 0x560486803980 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- 172.21.15.129:0/1683751270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 msgr2=0x5604868099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff168000f80 tx=0x7ff168009a20 comp rx=0 tx=0).stop 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- 172.21.15.129:0/1683751270 shutdown_connections 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56048680e240 0x560486819920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x5604868099a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- 172.21.15.129:0/1683751270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560486803060 0x560486805ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- 172.21.15.129:0/1683751270 >> 172.21.15.129:0/1683751270 conn(0x5604867ff5f0 msgr2=0x560486819210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- 172.21.15.129:0/1683751270 shutdown_connections 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- 172.21.15.129:0/1683751270 wait complete. 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- start start 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 Processor -- start 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x560486820d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x5604868217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781af6e0 con 0x560486803980 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781b0b50 con 0x560486803060 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.217+0000 7ff182a2a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff1781afdb0 con 0x56048680e240 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff182a2a740 10 client.0 _ll_register_callbacks cb 0x7ff1680098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:40.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56838/0 (socket says 172.21.15.129:56838) 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3431314696 learned_addr learned my addr 172.21.15.129:0/3431314696 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x5604868217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x560486820d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3431314696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 msgr2=0x5604868217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x5604868217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 msgr2=0x560486820d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x560486820d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 -- 172.21.15.129:0/3431314696 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56048680a080 con 0x560486803980 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff17e3ed640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff168030620 tx=0x7ff168003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44dafd03f6ef7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff15dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff168005bd0 con 0x560486803980 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff168030860 con 0x560486803980 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff168004b40 con 0x560486803980 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff182a2a740 10 client.16098 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.218+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560486888850 con 0x560486803980 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff168036040 con 0x560486803980 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff16806a030 con 0x560486803980 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 10 client.16098 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 10 client.16098 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 20 client.16098 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560486888d20 0x56048688b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 4 client.16098 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fcd4421c", "release": true, "await": true}] 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fcd4421c", "release": true, "await": true}) -- 0x5604867700a0 con 0x560486888d20 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560486888d20 0x56048688b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:40.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff15dffb640 10 client.16098 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:40.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.219+0000 7ff17dbec640 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560486888d20 0x56048688b110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af9e8e64e6bb3dd server_cookie=588e39dae0b92bbd in_seq=0 out_seq=0 2024-09-05T01:22:40.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff15dffb640 1 -- 172.21.15.129:0/3431314696 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604867700a0 con 0x560486888d20 2024-09-05T01:22:40.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff15dffb640 10 client.16098 handle_command_reply: tid=0 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 client.16098 shutdown 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 msgr2=0x560486821260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff168030620 tx=0x7ff168003c20 comp rx=0 tx=0).stop 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 shutdown_connections 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560486888d20 0x56048688b110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff15dffb640 0 client.16098 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56048680e240 0x5604868217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560486803980 0x560486821260 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 --2- 172.21.15.129:0/3431314696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560486803060 0x560486820d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 >> 172.21.15.129:0/3431314696 conn(0x5604867ff5f0 msgr2=0x560486819210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 shutdown_connections 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.276+0000 7ff182a2a740 1 -- 172.21.15.129:0/3431314696 wait complete. 2024-09-05T01:22:40.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff182a2a740 20 client.16098 trim_cache size 0 max 16384 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 24, 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "fcd4421c": { 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 24, 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:22:40.279 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:40.280 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 msgr2=0x7ff1600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 0x7ff1600a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff1700979c0 tx=0x7ff1700bb040 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 msgr2=0x7ff178074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff170009fd0 tx=0x7ff17000b8e0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 shutdown_connections 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1600a2b80 0x7ff1600a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff178106f20 0x7ff178074f70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1781027a0 0x7ff178074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 --2- 172.21.15.129:0/3874588562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff178101df0 0x7ff1780744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.277+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 >> 172.21.15.129:0/3874588562 conn(0x7ff1780fd8f0 msgr2=0x7ff1780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.278+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 shutdown_connections 2024-09-05T01:22:40.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:22:40.278+0000 7ff180651640 1 -- 172.21.15.129:0/3874588562 wait complete. 2024-09-05T01:22:41.059 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fcd4421c', total seconds elapsed: 13.4 2024-09-05T01:22:41.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:41 smithi129 ceph-mon[30979]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2024-09-05T01:22:41.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:41 smithi183 ceph-mon[34589]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2024-09-05T01:22:41.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:41 smithi194 ceph-mon[34941]: pgmap v572: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.9 MiB/s wr, 533 op/s 2024-09-05T01:22:43.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:43 smithi183 ceph-mon[34589]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-05T01:22:43.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:43 smithi129 ceph-mon[30979]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-05T01:22:43.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:43 smithi194 ceph-mon[34941]: pgmap v573: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 5.7 MiB/s wr, 433 op/s 2024-09-05T01:22:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 340.0 s, 1072.9 tps, lat 227.048 ms stddev 981.858 2024-09-05T01:22:45.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:45 smithi183 ceph-mon[34589]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-05T01:22:45.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:45 smithi129 ceph-mon[30979]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-05T01:22:45.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:45 smithi194 ceph-mon[34941]: pgmap v574: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-05T01:22:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 -- 172.21.15.129:0/3739800886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 msgr2=0x7fefac100210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/3739800886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac100210 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009ca0 tx=0x7fefa402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 -- 172.21.15.129:0/3739800886 shutdown_connections 2024-09-05T01:22:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/3739800886 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac101530 0x7fefac105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/3739800886 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac100b70 0x7fefac100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/3739800886 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac100210 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 -- 172.21.15.129:0/3739800886 >> 172.21.15.129:0/3739800886 conn(0x7fefac0fb970 msgr2=0x7fefac0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 -- 172.21.15.129:0/3739800886 shutdown_connections 2024-09-05T01:22:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.357+0000 7fefb1b6d640 1 -- 172.21.15.129:0/3739800886 wait complete. 2024-09-05T01:22:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.359+0000 7fefb1b6d640 1 Processor -- start 2024-09-05T01:22:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.359+0000 7fefb1b6d640 1 -- start start 2024-09-05T01:22:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 0x7fefac19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 0x7fefac1a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefac113d40 con 0x7fefac0ffe10 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefac113bc0 con 0x7fefac101530 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefaaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 0x7fefac19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefaaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 0x7fefac19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:33238/0 (socket says 172.21.15.129:33238) 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefaaffd640 1 -- 172.21.15.129:0/690664198 learned_addr learned my addr 172.21.15.129:0/690664198 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.360+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefac113ec0 con 0x7fefac100b70 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 0x7fefac1a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefab7fe640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 -- 172.21.15.129:0/690664198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 msgr2=0x7fefac19e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 0x7fefac19e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 -- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 msgr2=0x7fefac19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.361+0000 7fefabfff640 1 -- 172.21.15.129:0/690664198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac1a5f00 con 0x7fefac101530 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.362+0000 7fefabfff640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 0x7fefac1a5890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e970 tx=0x7fefa000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.363+0000 7fefa8ff9640 1 -- 172.21.15.129:0/690664198 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa000ccc0 con 0x7fefac101530 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.363+0000 7fefa8ff9640 1 -- 172.21.15.129:0/690664198 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa0005020 con 0x7fefac101530 2024-09-05T01:22:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.363+0000 7fefa8ff9640 1 -- 172.21.15.129:0/690664198 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0004a00 con 0x7fefac101530 2024-09-05T01:22:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.363+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac1a61f0 con 0x7fefac101530 2024-09-05T01:22:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.363+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1a66f0 con 0x7fefac101530 2024-09-05T01:22:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.367+0000 7fefb49ec740 1 Processor -- start 2024-09-05T01:22:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.367+0000 7fefa8ff9640 1 -- 172.21.15.129:0/690664198 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fefa00051d0 con 0x7fefac101530 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 -- start start 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d51c33d2f0 con 0x55d51c3d99a0 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d51c2e5270 con 0x55d51c3da2e0 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefb49ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d51c3392b0 con 0x55d51c3df370 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefaaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefaaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41720/0 (socket says 172.21.15.129:41720) 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefaaffd640 1 -- 172.21.15.129:0/120575242 learned_addr learned my addr 172.21.15.129:0/120575242 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.368+0000 7fefab7fe640 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 -- 172.21.15.129:0/120575242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 msgr2=0x55d51c3e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 -- 172.21.15.129:0/120575242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 msgr2=0x55d51c3deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 -- 172.21.15.129:0/120575242 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d51c2e4340 con 0x55d51c3d99a0 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefa8ff9640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefa8ff9640 1 -- 172.21.15.129:0/690664198 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fefa00cf3b0 con 0x7fefac101530 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefab7fe640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fefaaffd640 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00cdd0 tx=0x7fef9c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca202a1be2dbdfea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fef91ffb640 1 -- 172.21.15.129:0/120575242 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c005220 con 0x55d51c3d99a0 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.369+0000 7fef91ffb640 1 -- 172.21.15.129:0/120575242 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef9c004c50 con 0x55d51c3d99a0 2024-09-05T01:22:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.370+0000 7fef91ffb640 1 -- 172.21.15.129:0/120575242 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0128e0 con 0x55d51c3d99a0 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.370+0000 7fefb49ec740 1 -- 172.21.15.129:0/120575242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 msgr2=0x55d51c3d9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.370+0000 7fefb49ec740 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00cdd0 tx=0x7fef9c0117b0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 -- 172.21.15.129:0/120575242 shutdown_connections 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 --2- 172.21.15.129:0/120575242 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d9da0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 -- 172.21.15.129:0/120575242 >> 172.21.15.129:0/120575242 conn(0x55d51c3d5010 msgr2=0x55d51c3d5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefab7fe640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 0x7fef940a51c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefa4006010 tx=0x7fefa4031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 -- 172.21.15.129:0/120575242 shutdown_connections 2024-09-05T01:22:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.371+0000 7fefb49ec740 1 -- 172.21.15.129:0/120575242 wait complete. 2024-09-05T01:22:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.372+0000 7fefb49ec740 1 -- start start 2024-09-05T01:22:46.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.372+0000 7fefb49ec740 1 Processor -- start 2024-09-05T01:22:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3d99a0 0x55d51c3e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d51c3e23e0 con 0x55d51c3df370 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d51c2c6290 con 0x55d51c3da2e0 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d51c3e2560 con 0x55d51c3d99a0 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefb49ec740 10 client.0 _ll_register_callbacks cb 0x55d51c2f7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefabfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefabfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41732/0 (socket says 172.21.15.129:41732) 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.374+0000 7fefabfff640 1 -- 172.21.15.129:0/1770705025 learned_addr learned my addr 172.21.15.129:0/1770705025 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 -- 172.21.15.129:0/1770705025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3d99a0 msgr2=0x55d51c3e3c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3d99a0 0x55d51c3e3c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 -- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 msgr2=0x55d51c3e4180 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3e4180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 -- 172.21.15.129:0/1770705025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d51c050210 con 0x55d51c3df370 2024-09-05T01:22:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fefabfff640 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fefa0097c60 tx=0x7fefa0005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfff2313aeb6426 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.375+0000 7fef8affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.376+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa001c040 con 0x55d51c3df370 2024-09-05T01:22:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.376+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fefa00860f0 con 0x55d51c3df370 2024-09-05T01:22:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.376+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c004c50 con 0x55d51c3df370 2024-09-05T01:22:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.377+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa009fbb0 con 0x55d51c3df370 2024-09-05T01:22:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.377+0000 7fefb49ec740 10 client.16110 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.377+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d51c11c620 con 0x55d51c3df370 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.377+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa00a4470 con 0x55d51c3df370 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.377+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fefa009ac50 con 0x55d51c3df370 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 10 client.16110 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 20 client.16110 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c3ef8f0 0x55d51c3f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 4 client.16110 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.378+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fef9c004c50 con 0x55d51c3ef8f0 2024-09-05T01:22:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.379+0000 7fefab7fe640 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c3ef8f0 0x55d51c3f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.380+0000 7fefab7fe640 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c3ef8f0 0x55d51c3f1d80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430bd827f109fd49 server_cookie=5f5446a42e84016f in_seq=0 out_seq=0 2024-09-05T01:22:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.380+0000 7fef8affd640 10 client.16110 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.381+0000 7fef8affd640 1 -- 172.21.15.129:0/1770705025 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fef9c004c50 con 0x55d51c3ef8f0 2024-09-05T01:22:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.381+0000 7fef8affd640 10 client.16110 handle_command_reply: tid=0 2024-09-05T01:22:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.381+0000 7fefb49ec740 1 client.16110 shutdown 2024-09-05T01:22:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.383+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 msgr2=0x55d51c3eb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.383+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fefa0097c60 tx=0x7fefa0005620 comp rx=0 tx=0).stop 2024-09-05T01:22:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 shutdown_connections 2024-09-05T01:22:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c3ef8f0 0x55d51c3f1d80 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3df370 0x55d51c3eb1c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1770705025 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3d99a0 0x55d51c3e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 >> 172.21.15.129:0/1770705025 conn(0x55d51c3d5010 msgr2=0x55d51c3d6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fef8affd640 0 client.16110 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.384+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 shutdown_connections 2024-09-05T01:22:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.385+0000 7fefb49ec740 1 -- 172.21.15.129:0/1770705025 wait complete. 2024-09-05T01:22:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.385+0000 7fefb49ec740 20 client.16110 trim_cache size 0 max 16384 2024-09-05T01:22:46.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:46 smithi183 ceph-mon[34589]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 380 op/s 2024-09-05T01:22:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.413+0000 7fefb49ec740 1 Processor -- start 2024-09-05T01:22:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 -- start start 2024-09-05T01:22:46.414 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3d3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b3870 con 0x55d51c3d99a0 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b1140 con 0x55d51c3da2e0 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefb49ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1aef90 con 0x55d51c3df370 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41740/0 (socket says 172.21.15.129:41740) 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefab7fe640 1 -- 172.21.15.129:0/1418785326 learned_addr learned my addr 172.21.15.129:0/1418785326 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:46.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.414+0000 7fefabfff640 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3d3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 -- 172.21.15.129:0/1418785326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 msgr2=0x55d51c3eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 -- 172.21.15.129:0/1418785326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 msgr2=0x55d51c3d39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 -- 172.21.15.129:0/1418785326 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d51c3d4e30 con 0x55d51c3df370 2024-09-05T01:22:46.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.415+0000 7fefabfff640 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3d3ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefa0099ab0 tx=0x7fefa0099d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76350cdad0eb89cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.416+0000 7fef917fa640 1 -- 172.21.15.129:0/1418785326 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa001c030 con 0x55d51c3df370 2024-09-05T01:22:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.416+0000 7fef917fa640 1 -- 172.21.15.129:0/1418785326 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa00a4470 con 0x55d51c3df370 2024-09-05T01:22:46.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.416+0000 7fef917fa640 1 -- 172.21.15.129:0/1418785326 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa009ac50 con 0x55d51c3df370 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.417+0000 7fefb49ec740 1 -- 172.21.15.129:0/1418785326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 msgr2=0x55d51c3d3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.417+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3d3ee0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefa0099ab0 tx=0x7fefa0099d10 comp rx=0 tx=0).stop 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.417+0000 7fefb49ec740 1 -- 172.21.15.129:0/1418785326 shutdown_connections 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.417+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3d3ee0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3da2e0 0x55d51c3eb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.418 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1418785326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3d99a0 0x55d51c3d39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.419 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 -- 172.21.15.129:0/1418785326 >> 172.21.15.129:0/1418785326 conn(0x55d51c3d5010 msgr2=0x55d51c3d53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 -- 172.21.15.129:0/1418785326 shutdown_connections 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 -- 172.21.15.129:0/1418785326 wait complete. 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.418+0000 7fefb49ec740 1 -- start start 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 Processor -- start 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3d99a0 0x55d51c3e9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3da2e0 0x55d51c3e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1af600 con 0x55d51c3da2e0 2024-09-05T01:22:46.420 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b0a70 con 0x55d51c3d99a0 2024-09-05T01:22:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1afcd0 con 0x55d51c3df370 2024-09-05T01:22:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.419+0000 7fefb49ec740 10 client.0 _ll_register_callbacks cb 0x7fefa00973e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:46.421 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:33260/0 (socket says 172.21.15.129:33260) 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 -- 172.21.15.129:0/1812869885 learned_addr learned my addr 172.21.15.129:0/1812869885 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefab7fe640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3d99a0 0x55d51c3e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefaaffd640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3da2e0 0x55d51c3e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 -- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3d99a0 msgr2=0x55d51c3e9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3d99a0 0x55d51c3e9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 -- 172.21.15.129:0/1812869885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3da2e0 msgr2=0x55d51c3e96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3da2e0 0x55d51c3e96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.420+0000 7fefabfff640 1 -- 172.21.15.129:0/1812869885 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d51c3eacf0 con 0x55d51c3df370 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fefabfff640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefa0018000 tx=0x7fefa0004e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1793608b3b2aac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:46.422 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fef8affd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:22:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa001c070 con 0x55d51c3df370 2024-09-05T01:22:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fefa00a41f0 con 0x55d51c3df370 2024-09-05T01:22:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa00a4470 con 0x55d51c3df370 2024-09-05T01:22:46.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.421+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fefa009cb30 con 0x55d51c3df370 2024-09-05T01:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.423+0000 7fefb49ec740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.424+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d51c45af10 con 0x55d51c3df370 2024-09-05T01:22:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fefa00a4470 con 0x55d51c3df370 2024-09-05T01:22:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fefa009a8a0 con 0x55d51c3df370 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 20 client.25603 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c45b800 0x55d51c45dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 4 client.25603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d51c34aa20 con 0x55d51c45b800 2024-09-05T01:22:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.425+0000 7fefab7fe640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c45b800 0x55d51c45dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.426+0000 7fef8affd640 10 client.25603 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.426+0000 7fefab7fe640 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c45b800 0x55d51c45dbf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6b907d0207f0e74 server_cookie=a6f54152c354d1cb in_seq=0 out_seq=0 2024-09-05T01:22:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.458+0000 7fef8affd640 1 -- 172.21.15.129:0/1812869885 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d51c34aa20 con 0x55d51c45b800 2024-09-05T01:22:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.458+0000 7fef8affd640 10 client.25603 handle_command_reply: tid=0 2024-09-05T01:22:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.458+0000 7fefb49ec740 1 client.25603 shutdown 2024-09-05T01:22:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.458+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 msgr2=0x55d51c3ea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.458+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefa0018000 tx=0x7fefa0004e30 comp rx=0 tx=0).stop 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 shutdown_connections 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d51c45b800 0x55d51c45dbf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fef8affd640 0 client.25603 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d51c3df370 0x55d51c3ea6e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d51c3da2e0 0x55d51c3e96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 --2- 172.21.15.129:0/1812869885 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d51c3d99a0 0x55d51c3e9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 >> 172.21.15.129:0/1812869885 conn(0x55d51c3d5010 msgr2=0x55d51c3d5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.459+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 shutdown_connections 2024-09-05T01:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.461+0000 7fefb49ec740 1 -- 172.21.15.129:0/1812869885 wait complete. 2024-09-05T01:22:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.461+0000 7fefb49ec740 20 client.25603 trim_cache size 0 max 16384 2024-09-05T01:22:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:22:46.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:22:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:22:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:22:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 msgr2=0x7fef940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 0x7fef940a51c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefa4006010 tx=0x7fefa4031000 comp rx=0 tx=0).stop 2024-09-05T01:22:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 msgr2=0x7fefac1a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 0x7fefac1a5890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefa000e970 tx=0x7fefa000ee30 comp rx=0 tx=0).stop 2024-09-05T01:22:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 shutdown_connections 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fefac101530 0x7fefac1a5890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fefac100b70 0x7fefac19e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 --2- 172.21.15.129:0/690664198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fefac0ffe10 0x7fefac19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.467+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 >> 172.21.15.129:0/690664198 conn(0x7fefac0fb970 msgr2=0x7fefac0fc570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.468+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 shutdown_connections 2024-09-05T01:22:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:46.468+0000 7fefb1b6d640 1 -- 172.21.15.129:0/690664198 wait complete. 2024-09-05T01:22:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:46 smithi129 ceph-mon[30979]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 380 op/s 2024-09-05T01:22:46.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:46 smithi194 ceph-mon[34941]: pgmap v575: 129 pgs: 129 active+clean; 1.3 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.6 MiB/s wr, 380 op/s 2024-09-05T01:22:49.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:48 smithi129 ceph-mon[30979]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-09-05T01:22:49.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:48 smithi194 ceph-mon[34941]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-09-05T01:22:49.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:48 smithi183 ceph-mon[34589]: pgmap v576: 129 pgs: 129 active+clean; 1.3 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.0 MiB/s wr, 591 op/s 2024-09-05T01:22:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 345.0 s, 992.8 tps, lat 100.969 ms stddev 48.374 2024-09-05T01:22:51.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:50 smithi129 ceph-mon[30979]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 677 op/s 2024-09-05T01:22:51.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:50 smithi194 ceph-mon[34941]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 677 op/s 2024-09-05T01:22:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:50 smithi183 ceph-mon[34589]: pgmap v577: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 677 op/s 2024-09-05T01:22:53.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:52 smithi194 ceph-mon[34941]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-09-05T01:22:53.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:52 smithi183 ceph-mon[34589]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-09-05T01:22:53.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:52 smithi129 ceph-mon[30979]: pgmap v578: 129 pgs: 129 active+clean; 1.3 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.6 MiB/s wr, 729 op/s 2024-09-05T01:22:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 350.0 s, 902.2 tps, lat 110.405 ms stddev 56.519 2024-09-05T01:22:55.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:54 smithi129 ceph-mon[30979]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.0 MiB/s wr, 914 op/s 2024-09-05T01:22:55.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:54 smithi194 ceph-mon[34941]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.0 MiB/s wr, 914 op/s 2024-09-05T01:22:55.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:54 smithi183 ceph-mon[34589]: pgmap v579: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 9.0 MiB/s wr, 914 op/s 2024-09-05T01:22:57.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:57 smithi183 ceph-mon[34589]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 916 op/s 2024-09-05T01:22:57.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:57 smithi129 ceph-mon[30979]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 916 op/s 2024-09-05T01:22:57.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:57 smithi194 ceph-mon[34941]: pgmap v580: 129 pgs: 129 active+clean; 1.3 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 9.4 MiB/s wr, 916 op/s 2024-09-05T01:22:59.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:22:59 smithi183 ceph-mon[34589]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-09-05T01:22:59.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:22:59 smithi129 ceph-mon[30979]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-09-05T01:22:59.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:22:59 smithi194 ceph-mon[34941]: pgmap v581: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 11 MiB/s wr, 1.04k op/s 2024-09-05T01:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.628+0000 7fd477752640 1 -- 172.21.15.129:0/2311347992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109d20 msgr2=0x7fd470112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.628+0000 7fd477752640 1 --2- 172.21.15.129:0/2311347992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109d20 0x7fd470112540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd468009be0 tx=0x7fd46802a6d0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.631 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.2 2024-09-05T01:22:59.631 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:22:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.630+0000 7fd477752640 1 -- 172.21.15.129:0/2311347992 shutdown_connections 2024-09-05T01:22:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.630+0000 7fd477752640 1 --2- 172.21.15.129:0/2311347992 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109d20 0x7fd470112540 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.630+0000 7fd477752640 1 --2- 172.21.15.129:0/2311347992 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109280 0x7fd470109700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.630+0000 7fd477752640 1 --2- 172.21.15.129:0/2311347992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 0x7fd470073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.630+0000 7fd477752640 1 -- 172.21.15.129:0/2311347992 >> 172.21.15.129:0/2311347992 conn(0x7fd47006f640 msgr2=0x7fd470071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.631+0000 7fd477752640 1 -- 172.21.15.129:0/2311347992 shutdown_connections 2024-09-05T01:22:59.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.631+0000 7fd477752640 1 -- 172.21.15.129:0/2311347992 wait complete. 2024-09-05T01:22:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 Processor -- start 2024-09-05T01:22:59.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 -- start start 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 0x7fd4701194c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109280 0x7fd470119a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd470113cd0 con 0x7fd470073aa0 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd470113b50 con 0x7fd470109d20 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd477752640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd470113e50 con 0x7fd470109280 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd475cef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd475cef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47326/0 (socket says 172.21.15.129:47326) 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd475cef640 1 -- 172.21.15.129:0/3482206094 learned_addr learned my addr 172.21.15.129:0/3482206094 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.632+0000 7fd4754ee640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 0x7fd4701194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd474ced640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109280 0x7fd470119a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd475cef640 1 -- 172.21.15.129:0/3482206094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109280 msgr2=0x7fd470119a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd475cef640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109280 0x7fd470119a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd475cef640 1 -- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 msgr2=0x7fd4701194c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd475cef640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 0x7fd4701194c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.633+0000 7fd475cef640 1 -- 172.21.15.129:0/3482206094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd47011f0d0 con 0x7fd470109d20 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.635+0000 7fd475cef640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd468009b30 tx=0x7fd468003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.636+0000 7fd4667fc640 1 -- 172.21.15.129:0/3482206094 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468041020 con 0x7fd470109d20 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.636+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd47011f360 con 0x7fd470109d20 2024-09-05T01:22:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.636+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd47011f850 con 0x7fd470109d20 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.636+0000 7fd4667fc640 1 -- 172.21.15.129:0/3482206094 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd468037040 con 0x7fd470109d20 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.636+0000 7fd4667fc640 1 -- 172.21.15.129:0/3482206094 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468036070 con 0x7fd470109d20 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.640+0000 7fd47a5bf740 1 Processor -- start 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 -- start start 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db59da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 0x55fe1db61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe1dabd2f0 con 0x55fe1db599a0 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fe1da65270 con 0x55fe1db5a2e0 2024-09-05T01:22:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.641+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fe1dab92b0 con 0x55fe1db5f370 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd4667fc640 1 -- 172.21.15.129:0/3482206094 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd468044020 con 0x7fd470109d20 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd4754ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd4754ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47328/0 (socket says 172.21.15.129:47328) 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd4754ee640 1 -- 172.21.15.129:0/460042909 learned_addr learned my addr 172.21.15.129:0/460042909 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 0x55fe1db61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db59da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 -- 172.21.15.129:0/460042909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 msgr2=0x55fe1db5eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 -- 172.21.15.129:0/460042909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 msgr2=0x55fe1db59da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db59da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd474ced640 1 -- 172.21.15.129:0/460042909 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe1da64340 con 0x55fe1db5f370 2024-09-05T01:22:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.642+0000 7fd4754ee640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd474ced640 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 0x55fe1db61830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4600058c0 tx=0x7fd46000c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169b41fcfba5e2f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd4667fc640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 0x7fd45c0a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd45b7fe640 1 -- 172.21.15.129:0/460042909 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460004030 con 0x55fe1db5f370 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd45b7fe640 1 -- 172.21.15.129:0/460042909 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4600041d0 con 0x55fe1db5f370 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd45b7fe640 1 -- 172.21.15.129:0/460042909 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46000f750 con 0x55fe1db5f370 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.643+0000 7fd4754ee640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 0x7fd45c0a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.644+0000 7fd4667fc640 1 -- 172.21.15.129:0/3482206094 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fd4680f0250 con 0x7fd470109d20 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.644+0000 7fd47a5bf740 1 -- 172.21.15.129:0/460042909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 msgr2=0x55fe1db61830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.644+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 0x55fe1db61830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4600058c0 tx=0x7fd46000c650 comp rx=0 tx=0).stop 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 -- 172.21.15.129:0/460042909 shutdown_connections 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5f370 0x55fe1db61830 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4600058c0 tx=0x7fd46000c650 comp rx=0 tx=0).stop 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db5eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/460042909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db59da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 -- 172.21.15.129:0/460042909 >> 172.21.15.129:0/460042909 conn(0x55fe1db55010 msgr2=0x55fe1db55410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd4754ee640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 0x7fd45c0a5320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55fe1db5ad10 tx=0x7fd46c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 -- 172.21.15.129:0/460042909 shutdown_connections 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.645+0000 7fd47a5bf740 1 -- 172.21.15.129:0/460042909 wait complete. 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 -- start start 2024-09-05T01:22:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 Processor -- start 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db693e0 0x55fe1db6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fe1db62340 con 0x55fe1db693e0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fe1da46290 con 0x55fe1db5a2e0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fe1db624c0 con 0x55fe1db599a0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.646+0000 7fd47a5bf740 10 client.0 _ll_register_callbacks cb 0x55fe1da77140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60022/0 (socket says 172.21.15.129:60022) 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 -- 172.21.15.129:0/1121812254 learned_addr learned my addr 172.21.15.129:0/1121812254 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd474ced640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 -- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 msgr2=0x55fe1db643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 -- 172.21.15.129:0/1121812254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db693e0 msgr2=0x55fe1db6b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db693e0 0x55fe1db6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 -- 172.21.15.129:0/1121812254 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1d7d0210 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.647+0000 7fd4754ee640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd46c01ea00 tx=0x7fd46c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1468826f614b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.648+0000 7fd458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.648+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46c004020 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.648+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fd46c018030 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.648+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4600041d0 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd46c034050 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 10 client.25615 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe1d89c620 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4600041d0 con 0x55fe1db599a0 2024-09-05T01:22:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd46c00f480 con 0x55fe1db599a0 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 10 client.25615 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 20 client.25615 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1db6ffc0 0x55fe1db72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 4 client.25615 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.649+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe1dacaa20 con 0x55fe1db6ffc0 2024-09-05T01:22:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.651+0000 7fd474ced640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1db6ffc0 0x55fe1db72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.653+0000 7fd474ced640 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1db6ffc0 0x55fe1db72440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e379a6dc01908b10 server_cookie=b0c7856732f3074b in_seq=0 out_seq=0 2024-09-05T01:22:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.653+0000 7fd458ff9640 10 client.25615 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd458ff9640 1 -- 172.21.15.129:0/1121812254 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe1dacaa20 con 0x55fe1db6ffc0 2024-09-05T01:22:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd458ff9640 10 client.25615 handle_command_reply: tid=0 2024-09-05T01:22:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd47a5bf740 1 client.25615 shutdown 2024-09-05T01:22:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 msgr2=0x55fe1db63e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd46c01ea00 tx=0x7fd46c004730 comp rx=0 tx=0).stop 2024-09-05T01:22:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.655+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 shutdown_connections 2024-09-05T01:22:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.656+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1db6ffc0 0x55fe1db72440 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.656+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db693e0 0x55fe1db6b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.656+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db5a2e0 0x55fe1db643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.656+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/1121812254 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db599a0 0x55fe1db63e70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.656+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 >> 172.21.15.129:0/1121812254 conn(0x55fe1db55010 msgr2=0x55fe1db56820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.658+0000 7fd458ff9640 0 client.25615 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.658+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 shutdown_connections 2024-09-05T01:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.658+0000 7fd47a5bf740 1 -- 172.21.15.129:0/1121812254 wait complete. 2024-09-05T01:22:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.658+0000 7fd47a5bf740 20 client.25615 trim_cache size 0 max 16384 2024-09-05T01:22:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.689+0000 7fd47a5bf740 1 Processor -- start 2024-09-05T01:22:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.689+0000 7fd47a5bf740 1 -- start start 2024-09-05T01:22:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.689+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db64330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 0x55fe1db56400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd454012760 con 0x55fe1db599a0 2024-09-05T01:22:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd454010030 con 0x55fe1db693e0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd474ced640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd474ced640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60024/0 (socket says 172.21.15.129:60024) 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd474ced640 1 -- 172.21.15.129:0/562845286 learned_addr learned my addr 172.21.15.129:0/562845286 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd475cef640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 0x55fe1db56400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd45400de80 con 0x55fe1db5a2e0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.690+0000 7fd4754ee640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db64330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 -- 172.21.15.129:0/562845286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 msgr2=0x55fe1db55ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 -- 172.21.15.129:0/562845286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 msgr2=0x55fe1db64330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db64330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 -- 172.21.15.129:0/562845286 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe1db639b0 con 0x55fe1db693e0 2024-09-05T01:22:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd4754ee640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db64330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd474ced640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:22:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd475cef640 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 0x55fe1db56400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4680ed8e0 tx=0x7fd4680bc2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba06ad8fb0c1a7a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd45affd640 1 -- 172.21.15.129:0/562845286 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680d4070 con 0x55fe1db693e0 2024-09-05T01:22:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.691+0000 7fd45affd640 1 -- 172.21.15.129:0/562845286 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4680bcdc0 con 0x55fe1db693e0 2024-09-05T01:22:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.692+0000 7fd45affd640 1 -- 172.21.15.129:0/562845286 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468030a20 con 0x55fe1db693e0 2024-09-05T01:22:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.693+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 msgr2=0x55fe1db56400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.693+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 0x55fe1db56400 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4680ed8e0 tx=0x7fd4680bc2c0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.696+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 shutdown_connections 2024-09-05T01:22:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.696+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db693e0 0x55fe1db56400 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.696 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.696+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db5a2e0 0x55fe1db55ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.696+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/562845286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db599a0 0x55fe1db64330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.696+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 >> 172.21.15.129:0/562845286 conn(0x55fe1db55010 msgr2=0x55fe1db63c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.697+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 shutdown_connections 2024-09-05T01:22:59.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.697+0000 7fd47a5bf740 1 -- 172.21.15.129:0/562845286 wait complete. 2024-09-05T01:22:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.697+0000 7fd47a5bf740 1 -- start start 2024-09-05T01:22:59.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.698+0000 7fd47a5bf740 1 Processor -- start 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db599a0 0x55fe1db81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db693e0 0x55fe1db83160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd45400e4f0 con 0x55fe1db5a2e0 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd45400f960 con 0x55fe1db599a0 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd45400ebc0 con 0x55fe1db693e0 2024-09-05T01:22:59.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.701+0000 7fd47a5bf740 10 client.0 _ll_register_callbacks cb 0x7fd46c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:22:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.702+0000 7fd474ced640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd474ced640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60914/0 (socket says 172.21.15.129:60914) 2024-09-05T01:22:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd474ced640 1 -- 172.21.15.129:0/2400741564 learned_addr learned my addr 172.21.15.129:0/2400741564 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:22:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd475cef640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db693e0 0x55fe1db83160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd4754ee640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db599a0 0x55fe1db81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd474ced640 1 -- 172.21.15.129:0/2400741564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db693e0 msgr2=0x55fe1db83160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.703+0000 7fd474ced640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db693e0 0x55fe1db83160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.704+0000 7fd474ced640 1 -- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db599a0 msgr2=0x55fe1db81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.704+0000 7fd474ced640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db599a0 0x55fe1db81c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.704+0000 7fd474ced640 1 -- 172.21.15.129:0/2400741564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe1db83770 con 0x55fe1db5a2e0 2024-09-05T01:22:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.704+0000 7fd474ced640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd460004060 tx=0x7fd46000cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143e322fada926a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.704+0000 7fd458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.705+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460011e20 con 0x55fe1db5a2e0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.705+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4680bcdc0 con 0x55fe1db5a2e0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.705+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fd460004110 con 0x55fe1db5a2e0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.706+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd460004310 con 0x55fe1db5a2e0 2024-09-05T01:22:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.706+0000 7fd47a5bf740 10 client.16128 fetch_fsmap learned FSMap version 30 2024-09-05T01:22:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.706+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fe1dbdadb0 con 0x55fe1db5a2e0 2024-09-05T01:22:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd4680bcdc0 con 0x55fe1db5a2e0 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fd460004680 con 0x55fe1db5a2e0 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 10 client.16128 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 20 client.16128 populate_metadata read hostname 'smithi129' 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1dbdb600 0x55fe1dbdda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 4 client.16128 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe1dac4d70 con 0x55fe1dbdb600 2024-09-05T01:22:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.707+0000 7fd4754ee640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1dbdb600 0x55fe1dbdda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:22:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.708+0000 7fd458ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.708+0000 7fd4754ee640 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1dbdb600 0x55fe1dbdda20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa850ce56016430 server_cookie=ebd44bb5ce1f8700 in_seq=0 out_seq=0 2024-09-05T01:22:59.798 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.797+0000 7fd458ff9640 1 -- 172.21.15.129:0/2400741564 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe1dac4d70 con 0x55fe1dbdb600 2024-09-05T01:22:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.797+0000 7fd458ff9640 10 client.16128 handle_command_reply: tid=0 2024-09-05T01:22:59.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.798+0000 7fd47a5bf740 1 client.16128 shutdown 2024-09-05T01:22:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.800+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 msgr2=0x55fe1db82150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.800+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd460004060 tx=0x7fd46000cd90 comp rx=0 tx=0).stop 2024-09-05T01:22:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 shutdown_connections 2024-09-05T01:22:59.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fe1dbdb600 0x55fe1dbdda20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd458ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:22:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fe1db693e0 0x55fe1db83160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fe1db5a2e0 0x55fe1db82150 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 --2- 172.21.15.129:0/2400741564 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fe1db599a0 0x55fe1db81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.802+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 >> 172.21.15.129:0/2400741564 conn(0x55fe1db55010 msgr2=0x55fe1db539a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.803+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 shutdown_connections 2024-09-05T01:22:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.804+0000 7fd47a5bf740 1 -- 172.21.15.129:0/2400741564 wait complete. 2024-09-05T01:22:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.804+0000 7fd47a5bf740 20 client.16128 trim_cache size 0 max 16384 2024-09-05T01:22:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:22:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:22:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:22:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:22:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.807+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 msgr2=0x7fd45c0a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.807+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 0x7fd45c0a5320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55fe1db5ad10 tx=0x7fd46c009260 comp rx=0 tx=0).stop 2024-09-05T01:22:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 msgr2=0x7fd47011ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:22:59.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd468009b30 tx=0x7fd468003ba0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 shutdown_connections 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd45c0a2e60 0x7fd45c0a5320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd470109d20 0x7fd47011ea60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd470109280 0x7fd470119a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 --2- 172.21.15.129:0/3482206094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd470073aa0 0x7fd4701194c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 >> 172.21.15.129:0/3482206094 conn(0x7fd47006f640 msgr2=0x7fd470071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 shutdown_connections 2024-09-05T01:22:59.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:22:59.808+0000 7fd477752640 1 -- 172.21.15.129:0/3482206094 wait complete. 2024-09-05T01:23:00.187 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:00.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 355.0 s, 961.9 tps, lat 104.278 ms stddev 36.844 2024-09-05T01:23:01.340 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:01 smithi129 ceph-mon[30979]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 856 op/s 2024-09-05T01:23:01.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:01 smithi183 ceph-mon[34589]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 856 op/s 2024-09-05T01:23:01.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:01 smithi194 ceph-mon[34941]: pgmap v582: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.5 MiB/s wr, 856 op/s 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 -- 172.21.15.129:0/796532895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941011e0 msgr2=0x7f0294105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 --2- 172.21.15.129:0/796532895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941011e0 0x7f0294105980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0288009930 tx=0x7f028802a440 comp rx=0 tx=0).stop 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 -- 172.21.15.129:0/796532895 shutdown_connections 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 --2- 172.21.15.129:0/796532895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f0294106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 --2- 172.21.15.129:0/796532895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941011e0 0x7f0294105980 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 --2- 172.21.15.129:0/796532895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941008c0 0x7f0294100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.658+0000 7f029a5f2640 1 -- 172.21.15.129:0/796532895 >> 172.21.15.129:0/796532895 conn(0x7f02940fc7d0 msgr2=0x7f02940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.659+0000 7f029a5f2640 1 -- 172.21.15.129:0/796532895 shutdown_connections 2024-09-05T01:23:01.661 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.660+0000 7f029a5f2640 1 -- 172.21.15.129:0/796532895 wait complete. 2024-09-05T01:23:01.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.660+0000 7f029a5f2640 1 Processor -- start 2024-09-05T01:23:01.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 -- start start 2024-09-05T01:23:01.662 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f029419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0294118510 con 0x7f0294105ec0 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0294118390 con 0x7f02941011e0 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f029a5f2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f0294118690 con 0x7f02941008c0 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f0298b8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f029419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f0293fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:01.663 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f02937fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:01.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f0293fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60054/0 (socket says 172.21.15.129:60054) 2024-09-05T01:23:01.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f0298b8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f029419e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60934/0 (socket says 172.21.15.129:60934) 2024-09-05T01:23:01.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f0293fff640 1 -- 172.21.15.129:0/4267889594 learned_addr learned my addr 172.21.15.129:0/4267889594 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:01.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.661+0000 7f02937fe640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47356/0 (socket says 172.21.15.129:47356) 2024-09-05T01:23:01.664 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f02937fe640 1 -- 172.21.15.129:0/4267889594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 msgr2=0x7f02941a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f02937fe640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f02937fe640 1 -- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 msgr2=0x7f029419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f02937fe640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f029419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f02937fe640 1 -- 172.21.15.129:0/4267889594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029419f010 con 0x7f02941011e0 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.662+0000 7f0293fff640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.663+0000 7f02937fe640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f028802a960 tx=0x7f0288003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.664+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028803f070 con 0x7f02941011e0 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.664+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0288031d60 con 0x7f02941011e0 2024-09-05T01:23:01.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.664+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0288035030 con 0x7f02941011e0 2024-09-05T01:23:01.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.665+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029419f2a0 con 0x7f02941011e0 2024-09-05T01:23:01.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.665+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029419f700 con 0x7f02941011e0 2024-09-05T01:23:01.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.667+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0294105980 con 0x7f02941011e0 2024-09-05T01:23:01.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.669+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0288042070 con 0x7f02941011e0 2024-09-05T01:23:01.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.670+0000 7f02917fa640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 0x7f027c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:01.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.670+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f02880efbd0 con 0x7f02941011e0 2024-09-05T01:23:01.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.671+0000 7f0293fff640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 0x7f027c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:01.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.673+0000 7f0293fff640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 0x7f027c0a5210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0284005fd0 tx=0x7f028400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:01.677 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:01.676+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02880bc6c0 con 0x7f02941011e0 2024-09-05T01:23:02.065 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.065+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02941ab740 con 0x7f02941011e0 2024-09-05T01:23:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.068+0000 7f02917fa640 1 -- 172.21.15.129:0/4267889594 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f02880bbfc0 con 0x7f02941011e0 2024-09-05T01:23:02.069 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:23:02.069 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:23:02.069 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.072+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 msgr2=0x7f027c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.072+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 0x7f027c0a5210 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0284005fd0 tx=0x7f028400b480 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 msgr2=0x7f02941a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f028802a960 tx=0x7f0288003cd0 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 shutdown_connections 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f027c0a2d50 0x7f027c0a5210 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0294105ec0 0x7f029419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f02941011e0 0x7f02941a4c60 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f028802a960 tx=0x7f0288003cd0 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 --2- 172.21.15.129:0/4267889594 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f02941008c0 0x7f02941a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:02.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 >> 172.21.15.129:0/4267889594 conn(0x7f02940fc7d0 msgr2=0x7f0294103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:02.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 shutdown_connections 2024-09-05T01:23:02.075 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:02.073+0000 7f029a5f2640 1 -- 172.21.15.129:0/4267889594 wait complete. 2024-09-05T01:23:02.879 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 133.2 --await-for 45.0' 2024-09-05T01:23:02.880 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 133.2 --await-for 45.0 2024-09-05T01:23:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:03 smithi129 ceph-mon[30979]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:03 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4267889594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:03.301 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:03 smithi194 ceph-mon[34941]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:03.302 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:03 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4267889594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:03.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:03 smithi183 ceph-mon[34589]: pgmap v583: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:03 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4267889594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:03.448 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:04.086 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 -- 172.21.15.129:0/3336100782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 msgr2=0x7f6004108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 --2- 172.21.15.129:0/3336100782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f6004108830 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002a410 comp rx=0 tx=0).stop 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 -- 172.21.15.129:0/3336100782 shutdown_connections 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 --2- 172.21.15.129:0/3336100782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f600410d530 0x7f60041150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 --2- 172.21.15.129:0/3336100782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6004108e00 0x7f600410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 --2- 172.21.15.129:0/3336100782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f6004108830 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.085+0000 7f600c12b640 1 -- 172.21.15.129:0/3336100782 >> 172.21.15.129:0/3336100782 conn(0x7f6004077ba0 msgr2=0x7f6004079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.087+0000 7f600c12b640 1 -- 172.21.15.129:0/3336100782 shutdown_connections 2024-09-05T01:23:04.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.087+0000 7f600c12b640 1 -- 172.21.15.129:0/3336100782 wait complete. 2024-09-05T01:23:04.088 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.087+0000 7f600c12b640 1 Processor -- start 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 -- start start 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6004108e00 0x7f600406ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 0x7f600406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6004118430 con 0x7f600410d530 2024-09-05T01:23:04.089 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60041182b0 con 0x7f6004108e00 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600c12b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60041185b0 con 0x7f6004108450 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f6009ec7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f6009ec7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42408/0 (socket says 172.21.15.129:42408) 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f6009ec7640 1 -- 172.21.15.129:0/2653841003 learned_addr learned my addr 172.21.15.129:0/2653841003 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f60096c6640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6004108e00 0x7f600406ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.088+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 0x7f600406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.090 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 -- 172.21.15.129:0/2653841003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6004108e00 msgr2=0x7f600406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6004108e00 0x7f600406ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 -- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 msgr2=0x7f600406cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 0x7f600406cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 -- 172.21.15.129:0/2653841003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600406d550 con 0x7f6004108450 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 0x7f600406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f6009ec7640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6000005ab0 tx=0x7f6000003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.089+0000 7f5ffaffd640 1 -- 172.21.15.129:0/2653841003 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600003f070 con 0x7f6004108450 2024-09-05T01:23:04.091 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.090+0000 7f5ffaffd640 1 -- 172.21.15.129:0/2653841003 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6000031b50 con 0x7f6004108450 2024-09-05T01:23:04.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.091+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600406d7e0 con 0x7f6004108450 2024-09-05T01:23:04.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.091+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6004071770 con 0x7f6004108450 2024-09-05T01:23:04.092 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.091+0000 7f5ffaffd640 1 -- 172.21.15.129:0/2653841003 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000035030 con 0x7f6004108450 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.094+0000 7f600e504740 1 Processor -- start 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 -- start start 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e030b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e031080 0x555d0e03b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555d0df962a0 con 0x555d0e031080 2024-09-05T01:23:04.096 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d0df3db10 con 0x555d0e030760 2024-09-05T01:23:04.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.095+0000 7f600e504740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d0df92260 con 0x555d0e03b940 2024-09-05T01:23:04.097 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f60096c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f60096c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42412/0 (socket says 172.21.15.129:42412) 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f60096c6640 1 -- 172.21.15.129:0/2424000789 learned_addr learned my addr 172.21.15.129:0/2424000789 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f60096c6640 1 -- 172.21.15.129:0/2424000789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 msgr2=0x555d0e030b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e030b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.096+0000 7f5ffaffd640 1 -- 172.21.15.129:0/2653841003 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6000042070 con 0x7f6004108450 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f60096c6640 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e030b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f60096c6640 1 -- 172.21.15.129:0/2424000789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e031080 msgr2=0x555d0e03b400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:23:04.098 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f60096c6640 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e031080 0x555d0e03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f60096c6640 1 -- 172.21.15.129:0/2424000789 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0dd07b90 con 0x555d0e03b940 2024-09-05T01:23:04.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e030b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:23:04.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.097+0000 7f60096c6640 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400bc40 tx=0x7f5ff400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b7e834eafc89f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.099 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f5ffaffd640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 0x7f5ff00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f5feffff640 1 -- 172.21.15.129:0/2424000789 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4015070 con 0x555d0e03b940 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f5feffff640 1 -- 172.21.15.129:0/2424000789 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ff401a910 con 0x555d0e03b940 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f5feffff640 1 -- 172.21.15.129:0/2424000789 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4011900 con 0x555d0e03b940 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f5ffaffd640 1 -- 172.21.15.129:0/2653841003 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f60000efb20 con 0x7f6004108450 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.098+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 0x7f5ff00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.099+0000 7f600e504740 1 -- 172.21.15.129:0/2424000789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 msgr2=0x555d0e03dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.101 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.099+0000 7f600e504740 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400bc40 tx=0x7f5ff400f730 comp rx=0 tx=0).stop 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.099+0000 7f600a6c8640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 0x7f5ff00a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005fd0 tx=0x7f5ffc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.101+0000 7f600e504740 1 -- 172.21.15.129:0/2424000789 shutdown_connections 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.101+0000 7f600e504740 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e03dd30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.101+0000 7f600e504740 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e031080 0x555d0e03b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.101+0000 7f600e504740 1 --2- 172.21.15.129:0/2424000789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e030b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.101+0000 7f600e504740 1 -- 172.21.15.129:0/2424000789 >> 172.21.15.129:0/2424000789 conn(0x555d0e02ccf0 msgr2=0x555d0e02d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.102 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.102+0000 7f600e504740 1 -- 172.21.15.129:0/2424000789 shutdown_connections 2024-09-05T01:23:04.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.102+0000 7f600e504740 1 -- 172.21.15.129:0/2424000789 wait complete. 2024-09-05T01:23:04.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.103+0000 7f600e504740 1 -- start start 2024-09-05T01:23:04.103 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.103+0000 7f600e504740 1 Processor -- start 2024-09-05T01:23:04.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e030760 0x555d0e038900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.104 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e038e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e039380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555d0e04f520 con 0x555d0e030760 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d0df1e6e0 con 0x555d0e031080 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d0e04f6a0 con 0x555d0e03b940 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600a6c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e039380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600a6c8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e039380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42420/0 (socket says 172.21.15.129:42420) 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600a6c8640 1 -- 172.21.15.129:0/3521288696 learned_addr learned my addr 172.21.15.129:0/3521288696 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f600e504740 10 client.0 _ll_register_callbacks cb 0x555d0df42a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:04.105 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 -- 172.21.15.129:0/3521288696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 msgr2=0x555d0e039380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e039380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 -- 172.21.15.129:0/3521288696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e030760 msgr2=0x555d0e038900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e030760 0x555d0e038900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.104+0000 7f60096c6640 1 -- 172.21.15.129:0/3521288696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0e058b20 con 0x555d0e031080 2024-09-05T01:23:04.106 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f60096c6640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e038e40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000c00 tx=0x7f5ff401a690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40ccb8d07dc166d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f5fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:23:04.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4015070 con 0x555d0e031080 2024-09-05T01:23:04.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ff401a910 con 0x555d0e031080 2024-09-05T01:23:04.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f5ff401acd0 con 0x555d0e031080 2024-09-05T01:23:04.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.105+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ff4005040 con 0x555d0e031080 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.106+0000 7f600e504740 10 client.25685 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.106+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d0e05b440 con 0x555d0e031080 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.106+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ff4011680 con 0x555d0e031080 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.106+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5ff4004680 con 0x555d0e031080 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 10 client.25685 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 10 client.25685 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 20 client.25685 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e05cd10 0x555d0e05f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 4 client.25685 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:04.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5ff401a910 con 0x555d0e05cd10 2024-09-05T01:23:04.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.107+0000 7f6009ec7640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e05cd10 0x555d0e05f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.111+0000 7f5fed7fa640 10 client.25685 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:04.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.111+0000 7f6009ec7640 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e05cd10 0x555d0e05f1b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f91bcf713d56335 server_cookie=872ad5ea7ddb2b7a in_seq=0 out_seq=0 2024-09-05T01:23:04.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.112+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3521288696 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f5ff401a910 con 0x555d0e05cd10 2024-09-05T01:23:04.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.112+0000 7f5fed7fa640 10 client.25685 handle_command_reply: tid=0 2024-09-05T01:23:04.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.112+0000 7f600e504740 1 client.25685 shutdown 2024-09-05T01:23:04.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 msgr2=0x555d0e038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e038e40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000c00 tx=0x7f5ff401a690 comp rx=0 tx=0).stop 2024-09-05T01:23:04.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 shutdown_connections 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e05cd10 0x555d0e05f1b0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e03b940 0x555d0e039380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e038e40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 --2- 172.21.15.129:0/3521288696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e030760 0x555d0e038900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 >> 172.21.15.129:0/3521288696 conn(0x555d0e02ccf0 msgr2=0x555d0e032a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f5fed7fa640 0 client.25685 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 shutdown_connections 2024-09-05T01:23:04.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 1 -- 172.21.15.129:0/3521288696 wait complete. 2024-09-05T01:23:04.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.113+0000 7f600e504740 20 client.25685 trim_cache size 0 max 16384 2024-09-05T01:23:04.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.144+0000 7f600e504740 1 Processor -- start 2024-09-05T01:23:04.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.144+0000 7f600e504740 1 -- start start 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f600e504740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e0391f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f600e504740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 0x555d0e02bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f600e504740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e064ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f60096c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 0x555d0e02bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f60096c6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 0x555d0e02bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42424/0 (socket says 172.21.15.129:42424) 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f60096c6640 1 -- 172.21.15.129:0/224055348 learned_addr learned my addr 172.21.15.129:0/224055348 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:04.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f6009ec7640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e0391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f600a6c8640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e064ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.145+0000 7f600e504740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60041b7930 con 0x555d0e03b940 2024-09-05T01:23:04.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.146+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60041b5200 con 0x555d0e030760 2024-09-05T01:23:04.147 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.146+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60041098e0 con 0x555d0e031080 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.146+0000 7f6009ec7640 1 -- 172.21.15.129:0/224055348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 msgr2=0x555d0e02bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.146+0000 7f6009ec7640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 0x555d0e02bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.146+0000 7f6009ec7640 1 -- 172.21.15.129:0/224055348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 msgr2=0x555d0e064ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.147+0000 7f6009ec7640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e064ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.147+0000 7f6009ec7640 1 -- 172.21.15.129:0/224055348 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d0e028260 con 0x555d0e030760 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.147+0000 7f600a6c8640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e064ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.147+0000 7f6009ec7640 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e0391f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x555d0e02db50 tx=0x7f6000040fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78178640c081d538 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.147+0000 7f5fef7fe640 1 -- 172.21.15.129:0/224055348 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000d6070 con 0x555d0e030760 2024-09-05T01:23:04.148 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.148+0000 7f5fef7fe640 1 -- 172.21.15.129:0/224055348 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f60000bc670 con 0x555d0e030760 2024-09-05T01:23:04.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.148+0000 7f5fef7fe640 1 -- 172.21.15.129:0/224055348 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000049040 con 0x555d0e030760 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 msgr2=0x555d0e0391f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e0391f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x555d0e02db50 tx=0x7f6000040fc0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 shutdown_connections 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e064ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e031080 0x555d0e02bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 --2- 172.21.15.129:0/224055348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e030760 0x555d0e0391f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.149+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 >> 172.21.15.129:0/224055348 conn(0x555d0e02ccf0 msgr2=0x555d0e02b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.150+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 shutdown_connections 2024-09-05T01:23:04.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.150+0000 7f600e504740 1 -- 172.21.15.129:0/224055348 wait complete. 2024-09-05T01:23:04.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.151+0000 7f600e504740 1 -- start start 2024-09-05T01:23:04.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.151+0000 7f600e504740 1 Processor -- start 2024-09-05T01:23:04.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.152+0000 7f600e504740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 0x555d0e035a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.152+0000 7f600e504740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e035fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.153 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f6009ec7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 0x555d0e035a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f6009ec7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 0x555d0e035a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42426/0 (socket says 172.21.15.129:42426) 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f6009ec7640 1 -- 172.21.15.129:0/3515503073 learned_addr learned my addr 172.21.15.129:0/3515503073 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e0364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f60096c6640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e035fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6004072290 con 0x555d0e03b940 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f60041b4b30 con 0x555d0e031080 2024-09-05T01:23:04.154 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f60040826a0 con 0x555d0e030760 2024-09-05T01:23:04.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.153+0000 7f600e504740 10 client.0 _ll_register_callbacks cb 0x7f5ff400b750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:04.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 -- 172.21.15.129:0/3515503073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 msgr2=0x555d0e035a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 0x555d0e035a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.155 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 -- 172.21.15.129:0/3515503073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 msgr2=0x555d0e0364f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e0364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d0e0c6bf0 con 0x555d0e031080 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f60096c6640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e035fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000c00 tx=0x7f5ff401a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de72e6359907a247 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f5fed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3515503073 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4037060 con 0x555d0e031080 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.154+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3515503073 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f5ff4004890 con 0x555d0e031080 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.155+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60000bc670 con 0x555d0e031080 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.155+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3515503073 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5ff4015040 con 0x555d0e031080 2024-09-05T01:23:04.156 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.155+0000 7f600e504740 10 client.25691 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.155+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=30}) -- 0x555d0e0c7d60 con 0x555d0e031080 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3515503073 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f5ff40051e0 con 0x555d0e031080 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 10 client.25691 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 10 client.25691 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 20 client.25691 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e0c8390 0x555d0e0ca810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:04.157 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 4 client.25691 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.2, "await_for": 45.0}] 2024-09-05T01:23:04.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.2, "await_for": 45.0}) -- 0x7f60000bc670 con 0x555d0e0c8390 2024-09-05T01:23:04.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f6009ec7640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e0c8390 0x555d0e0ca810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:04.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f6009ec7640 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e0c8390 0x555d0e0ca810 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc96f3714239488a server_cookie=b53f06ac24a4b8d2 in_seq=0 out_seq=0 2024-09-05T01:23:04.158 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.156+0000 7f5fed7fa640 10 client.25691 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f5fed7fa640 1 -- 172.21.15.129:0/3515503073 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f60000bc670 con 0x555d0e0c8390 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f5fed7fa640 10 client.25691 handle_command_reply: tid=0 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 client.25691 shutdown 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 msgr2=0x555d0e035fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e035fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4000c00 tx=0x7f5ff401a780 comp rx=0 tx=0).stop 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 shutdown_connections 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d0e0c8390 0x555d0e0ca810 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f5fed7fa640 0 client.25691 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:04.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d0e03b940 0x555d0e0364f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d0e031080 0x555d0e035fb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 --2- 172.21.15.129:0/3515503073 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d0e030760 0x555d0e035a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 >> 172.21.15.129:0/3515503073 conn(0x555d0e02ccf0 msgr2=0x555d0e02be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 shutdown_connections 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 1 -- 172.21.15.129:0/3515503073 wait complete. 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.444+0000 7f600e504740 20 client.25691 trim_cache size 0 max 16384 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:23:04.447 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 26, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "26d479b5": { 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 26, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 133.2, 2024-09-05T01:23:04.448 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:04.449 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 msgr2=0x7f5ff00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 0x7f5ff00a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc005fd0 tx=0x7f5ffc009500 comp rx=0 tx=0).stop 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 msgr2=0x7f60040739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6000005ab0 tx=0x7f6000003c20 comp rx=0 tx=0).stop 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 shutdown_connections 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5ff00a2c00 0x7f5ff00a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f600410d530 0x7f600406cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6004108e00 0x7f600406ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 --2- 172.21.15.129:0/2653841003 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6004108450 0x7f60040739c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:04.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 >> 172.21.15.129:0/2653841003 conn(0x7f6004077ba0 msgr2=0x7f6004078eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:04.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 shutdown_connections 2024-09-05T01:23:04.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:04.445+0000 7f600c12b640 1 -- 172.21.15.129:0/2653841003 wait complete. 2024-09-05T01:23:05.099 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '26d479b5', quiesce took 5.5 seconds. Will release after: 7.699999999999999 2024-09-05T01:23:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 360.0 s, 685.3 tps, lat 116.341 ms stddev 49.276 2024-09-05T01:23:05.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:05 smithi183 ceph-mon[34589]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-05T01:23:05.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:05 smithi129 ceph-mon[30979]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-05T01:23:05.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:05 smithi194 ceph-mon[34941]: pgmap v584: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 10 MiB/s wr, 1.00k op/s 2024-09-05T01:23:07.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:07 smithi183 ceph-mon[34589]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.0 MiB/s wr, 891 op/s 2024-09-05T01:23:07.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:07 smithi129 ceph-mon[30979]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.0 MiB/s wr, 891 op/s 2024-09-05T01:23:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:07 smithi194 ceph-mon[34941]: pgmap v585: 129 pgs: 129 active+clean; 1.3 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 9.0 MiB/s wr, 891 op/s 2024-09-05T01:23:09.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:09 smithi183 ceph-mon[34589]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.6 MiB/s wr, 875 op/s 2024-09-05T01:23:09.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:09 smithi129 ceph-mon[30979]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.6 MiB/s wr, 875 op/s 2024-09-05T01:23:09.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:09 smithi194 ceph-mon[34941]: pgmap v586: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.6 MiB/s wr, 875 op/s 2024-09-05T01:23:10.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 365.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:23:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:11 smithi183 ceph-mon[34589]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-09-05T01:23:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:11 smithi129 ceph-mon[30979]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-09-05T01:23:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:11 smithi194 ceph-mon[34941]: pgmap v587: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.3 MiB/s wr, 661 op/s 2024-09-05T01:23:12.800 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 -- 172.21.15.129:0/1088831806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf68107820 msgr2=0x7faf68109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1088831806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf68107820 0x7faf68109c90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ca0 tx=0x7faf6002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 -- 172.21.15.129:0/1088831806 shutdown_connections 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1088831806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf68107820 0x7faf68109c90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1088831806 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68104ef0 0x7faf681072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1088831806 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf681025c0 0x7faf681049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 -- 172.21.15.129:0/1088831806 >> 172.21.15.129:0/1088831806 conn(0x7faf680fc1e0 msgr2=0x7faf680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 -- 172.21.15.129:0/1088831806 shutdown_connections 2024-09-05T01:23:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 -- 172.21.15.129:0/1088831806 wait complete. 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.913+0000 7faf6df2c640 1 Processor -- start 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 -- start start 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf68104ef0 0x7faf68100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68107820 0x7faf68100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810d6b0 con 0x7faf681025c0 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810d530 con 0x7faf68104ef0 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf6df2c640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf6810d830 con 0x7faf68107820 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35574/0 (socket says 172.21.15.129:35574) 2024-09-05T01:23:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 -- 172.21.15.129:0/1741186835 learned_addr learned my addr 172.21.15.129:0/1741186835 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf67fff640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68107820 0x7faf68100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf68104ef0 0x7faf68100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 -- 172.21.15.129:0/1741186835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68107820 msgr2=0x7faf68100ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68107820 0x7faf68100ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 -- 172.21.15.129:0/1741186835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf68104ef0 msgr2=0x7faf68100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf68104ef0 0x7faf68100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.914+0000 7faf677fe640 1 -- 172.21.15.129:0/1741186835 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf68101340 con 0x7faf681025c0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf677fe640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00d950 tx=0x7faf5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf64ff9640 1 -- 172.21.15.129:0/1741186835 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c017070 con 0x7faf681025c0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf64ff9640 1 -- 172.21.15.129:0/1741186835 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf5c009c30 con 0x7faf681025c0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf681a1e60 con 0x7faf681025c0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf64ff9640 1 -- 172.21.15.129:0/1741186835 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00f620 con 0x7faf681025c0 2024-09-05T01:23:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.915+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf681a2270 con 0x7faf681025c0 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.917+0000 7faf64ff9640 1 -- 172.21.15.129:0/1741186835 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7faf5c0109a0 con 0x7faf681025c0 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 Processor -- start 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 -- start start 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 0x55755e8a7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55755e80b2f0 con 0x55755e8ad370 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55755e7b3270 con 0x55755e8a82e0 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf70cbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55755e8072b0 con 0x55755e8a79a0 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35576/0 (socket says 172.21.15.129:35576) 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 -- 172.21.15.129:0/1211025365 learned_addr learned my addr 172.21.15.129:0/1211025365 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf677fe640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf67fff640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 0x55755e8a7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf64ff9640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 0x7faf500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 -- 172.21.15.129:0/1211025365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 msgr2=0x55755e8a7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 0x55755e8a7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 -- 172.21.15.129:0/1211025365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 msgr2=0x55755e8acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf66ffd640 1 -- 172.21.15.129:0/1211025365 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755e7b2340 con 0x55755e8ad370 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf67fff640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 0x55755e8a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf67fff640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 0x7faf500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.918+0000 7faf64ff9640 1 -- 172.21.15.129:0/1741186835 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf5c0cf0d0 con 0x7faf681025c0 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.919+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7faf5800cb90 tx=0x7faf5800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca0dca277a279e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.919+0000 7faf4dffb640 1 -- 172.21.15.129:0/1211025365 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58010630 con 0x55755e8ad370 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.919+0000 7faf67fff640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 0x7faf500a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf60031aa0 tx=0x7faf600095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.919+0000 7faf4dffb640 1 -- 172.21.15.129:0/1211025365 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58010c80 con 0x55755e8ad370 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.919+0000 7faf4dffb640 1 -- 172.21.15.129:0/1211025365 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5800fa50 con 0x55755e8ad370 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- 172.21.15.129:0/1211025365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 msgr2=0x55755e8af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7faf5800cb90 tx=0x7faf5800e720 comp rx=0 tx=0).stop 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- 172.21.15.129:0/1211025365 shutdown_connections 2024-09-05T01:23:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8af830 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1211025365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a79a0 0x55755e8a7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- 172.21.15.129:0/1211025365 >> 172.21.15.129:0/1211025365 conn(0x55755e8a3010 msgr2=0x55755e8a3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- 172.21.15.129:0/1211025365 shutdown_connections 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- 172.21.15.129:0/1211025365 wait complete. 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- start start 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 Processor -- start 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.920+0000 7faf70cbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55755e8b0340 con 0x55755e8a79a0 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55755e794290 con 0x55755e8a82e0 2024-09-05T01:23:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55755e8b04c0 con 0x55755e8ad370 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 10 client.0 _ll_register_callbacks cb 0x55755e7c5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35580/0 (socket says 172.21.15.129:35580) 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 -- 172.21.15.129:0/1279840281 learned_addr learned my addr 172.21.15.129:0/1279840281 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf67fff640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 -- 172.21.15.129:0/1279840281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 msgr2=0x55755e8b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 -- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 msgr2=0x55755e8b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 -- 172.21.15.129:0/1279840281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755e51e210 con 0x55755e8a79a0 2024-09-05T01:23:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf677fe640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faf5c000c00 tx=0x7faf5c0d27a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87095cf6d384d74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c020070 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf5c097270 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf58010c80 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf5c09ae80 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 10 client.16155 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.921+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55755e5ea620 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58010c80 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf5c0d1110 con 0x55755e8a79a0 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 10 client.16155 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 20 client.16155 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e8bd7e0 0x55755e8bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 4 client.16155 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55755e818a20 con 0x55755e8bd7e0 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e8bd7e0 0x55755e8bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf46ffd640 10 client.16155 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.922+0000 7faf66ffd640 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e8bd7e0 0x55755e8bfc10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d86ca98e2cdfe1 server_cookie=4bae450e0db77446 in_seq=0 out_seq=0 2024-09-05T01:23:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf46ffd640 1 -- 172.21.15.129:0/1279840281 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55755e818a20 con 0x55755e8bd7e0 2024-09-05T01:23:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf46ffd640 10 client.16155 handle_command_reply: tid=0 2024-09-05T01:23:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 client.16155 shutdown 2024-09-05T01:23:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 msgr2=0x55755e8b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7faf5c000c00 tx=0x7faf5c0d27a0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 shutdown_connections 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e8bd7e0 0x55755e8bfc10 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.923+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf46ffd640 0 client.16155 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a82e0 0x55755e8b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 1 --2- 172.21.15.129:0/1279840281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a79a0 0x55755e8b1bc0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 >> 172.21.15.129:0/1279840281 conn(0x55755e8a3010 msgr2=0x55755e8a4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 shutdown_connections 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 1 -- 172.21.15.129:0/1279840281 wait complete. 2024-09-05T01:23:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.924+0000 7faf70cbd740 20 client.16155 trim_cache size 0 max 16384 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 Processor -- start 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 -- start start 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf681af310 con 0x55755e8a82e0 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf681acbe0 con 0x55755e8a79a0 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf70cbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf681aaa30 con 0x55755e8ad370 2024-09-05T01:23:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf67fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf66ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35586/0 (socket says 172.21.15.129:35586) 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.939+0000 7faf66ffd640 1 -- 172.21.15.129:0/2519746034 learned_addr learned my addr 172.21.15.129:0/2519746034 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf677fe640 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 -- 172.21.15.129:0/2519746034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 msgr2=0x55755e8a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 -- 172.21.15.129:0/2519746034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 msgr2=0x55755e8a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 -- 172.21.15.129:0/2519746034 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755e8a87c0 con 0x55755e8a82e0 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf66ffd640 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faf5800cb90 tx=0x7faf5800f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0aa0908f3c9ae8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf4d7fa640 1 -- 172.21.15.129:0/2519746034 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580052c0 con 0x55755e8a82e0 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf4d7fa640 1 -- 172.21.15.129:0/2519746034 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58005460 con 0x55755e8a82e0 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.940+0000 7faf4d7fa640 1 -- 172.21.15.129:0/2519746034 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580048a0 con 0x55755e8a82e0 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- 172.21.15.129:0/2519746034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 msgr2=0x55755e8b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faf5800cb90 tx=0x7faf5800f5c0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- 172.21.15.129:0/2519746034 shutdown_connections 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8ad370 0x55755e8a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8a82e0 0x55755e8b90d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 --2- 172.21.15.129:0/2519746034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- 172.21.15.129:0/2519746034 >> 172.21.15.129:0/2519746034 conn(0x55755e8a3010 msgr2=0x55755e8b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- 172.21.15.129:0/2519746034 shutdown_connections 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- 172.21.15.129:0/2519746034 wait complete. 2024-09-05T01:23:12.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 -- start start 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.941+0000 7faf70cbd740 1 Processor -- start 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8b66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a82e0 0x55755e8b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf681ab0a0 con 0x55755e8ad370 2024-09-05T01:23:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faf681ac510 con 0x55755e8a79a0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7faf681ab770 con 0x55755e8a82e0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf70cbd740 10 client.0 _ll_register_callbacks cb 0x7faf5c097120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35592/0 (socket says 172.21.15.129:35592) 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf67fff640 1 -- 172.21.15.129:0/4114896736 learned_addr learned my addr 172.21.15.129:0/4114896736 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8b66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf66ffd640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a82e0 0x55755e8b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 -- 172.21.15.129:0/4114896736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a82e0 msgr2=0x55755e8b6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a82e0 0x55755e8b6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 -- 172.21.15.129:0/4114896736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 msgr2=0x55755e8b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 -- 172.21.15.129:0/4114896736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55755e8b8250 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf677fe640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8b66f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0997d0 tx=0x7faf5c097dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ebf7dc440ef271 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.942+0000 7faf46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c014020 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7faf5c09ab70 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf58005460 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faf5c020040 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 10 client.25709 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55755e9284f0 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faf58005460 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7faf5c084ab0 con 0x55755e8a79a0 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 10 client.25709 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 10 client.25709 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 20 client.25709 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e928d20 0x55755e92b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 4 client.25709 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.943+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55755e812d70 con 0x55755e928d20 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.944+0000 7faf66ffd640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e928d20 0x55755e92b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.944+0000 7faf66ffd640 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e928d20 0x55755e92b140 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50fafe215899f3d9 server_cookie=14be97b506b9d040 in_seq=0 out_seq=0 2024-09-05T01:23:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.944+0000 7faf46ffd640 10 client.25709 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf46ffd640 1 -- 172.21.15.129:0/4114896736 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55755e812d70 con 0x55755e928d20 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf46ffd640 10 client.25709 handle_command_reply: tid=0 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 client.25709 shutdown 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 msgr2=0x55755e8b66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8b66f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0997d0 tx=0x7faf5c097dc0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 shutdown_connections 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55755e928d20 0x55755e92b140 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55755e8ad370 0x55755e8b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf46ffd640 0 client.25709 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55755e8a82e0 0x55755e8b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.962+0000 7faf70cbd740 1 --2- 172.21.15.129:0/4114896736 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55755e8a79a0 0x55755e8b66f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 >> 172.21.15.129:0/4114896736 conn(0x55755e8a3010 msgr2=0x55755e8a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 shutdown_connections 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf70cbd740 1 -- 172.21.15.129:0/4114896736 wait complete. 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf70cbd740 20 client.25709 trim_cache size 0 max 16384 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:23:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:23:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 msgr2=0x7faf500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 0x7faf500a50f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7faf60031aa0 tx=0x7faf600095c0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 msgr2=0x7faf68101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00d950 tx=0x7faf5c00de10 comp rx=0 tx=0).stop 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 shutdown_connections 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7faf500a2c30 0x7faf500a50f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7faf68107820 0x7faf68100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faf68104ef0 0x7faf68100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 --2- 172.21.15.129:0/1741186835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf681025c0 0x7faf68101f30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 >> 172.21.15.129:0/1741186835 conn(0x7faf680fc1e0 msgr2=0x7faf68102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 shutdown_connections 2024-09-05T01:23:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:12.963+0000 7faf6df2c640 1 -- 172.21.15.129:0/1741186835 wait complete. 2024-09-05T01:23:13.078 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:13.198 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:13 smithi129 ceph-mon[30979]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-09-05T01:23:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:13 smithi183 ceph-mon[34589]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-09-05T01:23:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:13 smithi194 ceph-mon[34941]: pgmap v588: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 5.3 MiB/s wr, 552 op/s 2024-09-05T01:23:13.602 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1045191605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94102570 msgr2=0x7f4f941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1045191605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94102570 0x7f4f941029b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009900 tx=0x7f4f8002a480 comp rx=0 tx=0).stop 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1045191605 shutdown_connections 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1045191605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f94102ef0 0x7f4f9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1045191605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94102570 0x7f4f941029b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1045191605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94100e20 0x7f4f94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1045191605 >> 172.21.15.129:0/1045191605 conn(0x7f4f940fc7d0 msgr2=0x7f4f940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:13.603 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1045191605 shutdown_connections 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.587+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1045191605 wait complete. 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 Processor -- start 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 -- start start 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f94102570 0x7f4f941989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 0x7f4f9419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94111fa0 con 0x7f4f94100e20 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94111e20 con 0x7f4f94102570 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f9ac34640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94112120 con 0x7f4f94102ef0 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:13.604 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35604/0 (socket says 172.21.15.129:35604) 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 -- 172.21.15.129:0/1874955941 learned_addr learned my addr 172.21.15.129:0/1874955941 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f991d1640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 0x7f4f9419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 -- 172.21.15.129:0/1874955941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 msgr2=0x7f4f9419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 0x7f4f9419cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 -- 172.21.15.129:0/1874955941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f94102570 msgr2=0x7f4f941989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f94102570 0x7f4f941989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f989d0640 1 -- 172.21.15.129:0/1874955941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9419d430 con 0x7f4f94100e20 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.588+0000 7f4f991d1640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 0x7f4f9419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f989d0640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4a0 tx=0x7f4f8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:13.605 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88014070 con 0x7f4f94100e20 2024-09-05T01:23:13.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9419d720 con 0x7f4f94100e20 2024-09-05T01:23:13.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f88010ac0 con 0x7f4f94100e20 2024-09-05T01:23:13.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8801e060 con 0x7f4f94100e20 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.589+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941a5000 con 0x7f4f94100e20 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.590+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f94101200 con 0x7f4f94100e20 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.591+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f4f88017070 con 0x7f4f94100e20 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.592+0000 7f4f91ffb640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 0x7f4f840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.592+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4f880cec30 con 0x7f4f94100e20 2024-09-05T01:23:13.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.592+0000 7f4f93fff640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 0x7f4f840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:13.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.592+0000 7f4f93fff640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 0x7f4f840a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f8002a9a0 tx=0x7f4f80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:13.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.596+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f8809b780 con 0x7f4f94100e20 2024-09-05T01:23:13.797 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.796+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f94109090 con 0x7f4f94100e20 2024-09-05T01:23:13.798 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.797+0000 7f4f91ffb640 1 -- 172.21.15.129:0/1874955941 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f4f8809b080 con 0x7f4f94100e20 2024-09-05T01:23:13.798 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:23:13.799 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:23:13.799 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:23:13.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.799+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 msgr2=0x7f4f840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.799+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 0x7f4f840a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f8002a9a0 tx=0x7f4f80000bc0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.799+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 msgr2=0x7f4f94198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.799+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f8800b4a0 tx=0x7f4f8800b960 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.799+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 shutdown_connections 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f4f840a2b80 0x7f4f840a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4f94102ef0 0x7f4f9419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f94102570 0x7f4f941989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 --2- 172.21.15.129:0/1874955941 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f94100e20 0x7f4f94198480 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 >> 172.21.15.129:0/1874955941 conn(0x7f4f940fc7d0 msgr2=0x7f4f940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 shutdown_connections 2024-09-05T01:23:13.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:13.800+0000 7f4f9ac34640 1 -- 172.21.15.129:0/1874955941 wait complete. 2024-09-05T01:23:14.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:14 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1874955941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.218 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:14 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:14 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1874955941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:14 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.494 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 26d479b5 --release --await' 2024-09-05T01:23:14.494 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 26d479b5 --release --await 2024-09-05T01:23:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:14 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1874955941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:14 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:23:14.756 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:15.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:15 smithi129 ceph-mon[30979]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2024-09-05T01:23:15.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:23:15.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:23:15.181 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:23:15.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 370.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 -- 172.21.15.129:0/696384789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78981091f0 msgr2=0x7f78981095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 --2- 172.21.15.129:0/696384789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78981091f0 0x7f78981095d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7894009900 tx=0x7f789402a4b0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 -- 172.21.15.129:0/696384789 shutdown_connections 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 --2- 172.21.15.129:0/696384789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7898114ab0 0x7f7898116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 --2- 172.21.15.129:0/696384789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898109b10 0x7f7898114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 --2- 172.21.15.129:0/696384789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f78981091f0 0x7f78981095d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.270+0000 7f78a06da640 1 -- 172.21.15.129:0/696384789 >> 172.21.15.129:0/696384789 conn(0x7f789806e140 msgr2=0x7f789806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.271+0000 7f78a06da640 1 -- 172.21.15.129:0/696384789 shutdown_connections 2024-09-05T01:23:15.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.271+0000 7f78a06da640 1 -- 172.21.15.129:0/696384789 wait complete. 2024-09-05T01:23:15.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 Processor -- start 2024-09-05T01:23:15.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 -- start start 2024-09-05T01:23:15.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78981091f0 0x7f78981c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898114ab0 0x7f78981c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7898118470 con 0x7f78981091f0 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f78981182f0 con 0x7f7898114ab0 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.272+0000 7f78a06da640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f78981185f0 con 0x7f7898109b10 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789ec77640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898114ab0 0x7f78981c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789e476640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78981091f0 0x7f78981c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:50170/0 (socket says 172.21.15.129:50170) 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 -- 172.21.15.129:0/184766535 learned_addr learned my addr 172.21.15.129:0/184766535 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 -- 172.21.15.129:0/184766535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898114ab0 msgr2=0x7f78981c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898114ab0 0x7f78981c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 -- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78981091f0 msgr2=0x7f78981c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78981091f0 0x7f78981c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 -- 172.21.15.129:0/184766535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78981c0fb0 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f789dc75640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e370 tx=0x7f788c00e830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f788b7fe640 1 -- 172.21.15.129:0/184766535 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c0134e0 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.273+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78981c12a0 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.274+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78981cd4f0 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.275+0000 7f788b7fe640 1 -- 172.21.15.129:0/184766535 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f788c013680 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.275+0000 7f788b7fe640 1 -- 172.21.15.129:0/184766535 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c0116d0 con 0x7f7898109b10 2024-09-05T01:23:15.288 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.276+0000 7f78a2ab3740 1 Processor -- start 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 -- start start 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a665440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a672630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55568a5ca620 con 0x55568a670240 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55568a571e50 con 0x55568a665060 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55568a5c65e0 con 0x55568a665980 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:50172/0 (socket says 172.21.15.129:50172) 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 -- 172.21.15.129:0/2918477932 learned_addr learned my addr 172.21.15.129:0/2918477932 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789ec77640 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a665440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.289 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 -- 172.21.15.129:0/2918477932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 msgr2=0x55568a665440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a665440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 -- 172.21.15.129:0/2918477932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 msgr2=0x55568a672630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a672630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789e476640 1 -- 172.21.15.129:0/2918477932 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568a562d40 con 0x55568a665980 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.277+0000 7f789ec77640 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a665440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f789e476640 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7894009900 tx=0x7f7894005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7493f47068dbf9d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f7888ff9640 1 -- 172.21.15.129:0/2918477932 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894040070 con 0x55568a665980 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f788b7fe640 1 -- 172.21.15.129:0/184766535 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f788c011870 con 0x7f7898109b10 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f7888ff9640 1 -- 172.21.15.129:0/2918477932 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7894004040 con 0x55568a665980 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f788b7fe640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 0x7f78840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f7888ff9640 1 -- 172.21.15.129:0/2918477932 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894035040 con 0x55568a665980 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2918477932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 msgr2=0x55568a66fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.278+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7894009900 tx=0x7f7894005bd0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2918477932 shutdown_connections 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a672630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a66fd00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2918477932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a665440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2918477932 >> 172.21.15.129:0/2918477932 conn(0x55568a6615f0 msgr2=0x55568a6619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f788b7fe640 1 -- 172.21.15.129:0/184766535 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f788c0cf990 con 0x7f7898109b10 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f789ec77640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 0x7f78840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2918477932 shutdown_connections 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2918477932 wait complete. 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f789ec77640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 0x7f78840a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f789806b430 tx=0x7f789000f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 -- start start 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.279+0000 7f78a2ab3740 1 Processor -- start 2024-09-05T01:23:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a66d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a66db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55568a672eb0 con 0x55568a665980 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55568a552f10 con 0x55568a670240 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55568a673030 con 0x55568a665060 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f78a2ab3740 10 client.0 _ll_register_callbacks cb 0x55568a576db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35640/0 (socket says 172.21.15.129:35640) 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 -- 172.21.15.129:0/2374114016 learned_addr learned my addr 172.21.15.129:0/2374114016 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789e476640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a66d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789ec77640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a66db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 -- 172.21.15.129:0/2374114016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 msgr2=0x55568a66d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a66d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 -- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 msgr2=0x55568a66db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a66db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 -- 172.21.15.129:0/2374114016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568a2d9580 con 0x55568a665980 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789e476640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a66d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f789dc75640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f788c013fd0 tx=0x7f788c09be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a342a2c64a9a4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f7881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.280+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c021040 con 0x55568a665980 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7894004040 con 0x55568a665980 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f788c09a130 con 0x55568a665980 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f788c0a4560 con 0x55568a665980 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 10 client.16176 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55568a3a3960 con 0x55568a665980 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7894004040 con 0x55568a665980 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f788c0b2030 con 0x55568a665980 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 10 client.16176 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 10 client.16176 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 20 client.16176 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a682300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 4 client.16176 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55568a5d7b90 con 0x55568a67fec0 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.281+0000 7f789e476640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a682300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.282+0000 7f7881ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.282+0000 7f789e476640 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a682300 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f98381706c06b3 server_cookie=b5e7c11c2cd22f85 in_seq=0 out_seq=0 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.282+0000 7f7881ffb640 1 -- 172.21.15.129:0/2374114016 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55568a5d7b90 con 0x55568a67fec0 2024-09-05T01:23:15.294 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.282+0000 7f7881ffb640 10 client.16176 handle_command_reply: tid=0 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.282+0000 7f78a2ab3740 1 client.16176 shutdown 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 msgr2=0x55568a66d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f788c013fd0 tx=0x7f788c09be30 comp rx=0 tx=0).stop 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 shutdown_connections 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a682300 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a66db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f7881ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a66d610 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2374114016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a66d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 >> 172.21.15.129:0/2374114016 conn(0x55568a6615f0 msgr2=0x55568a6672d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 shutdown_connections 2024-09-05T01:23:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2374114016 wait complete. 2024-09-05T01:23:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.283+0000 7f78a2ab3740 20 client.16176 trim_cache size 0 max 16384 2024-09-05T01:23:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 Processor -- start 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 -- start start 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a65c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a6671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f787c0125c0 con 0x55568a670240 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00fe90 con 0x55568a665060 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00dce0 con 0x55568a665980 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35642/0 (socket says 172.21.15.129:35642) 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 -- 172.21.15.129:0/926362286 learned_addr learned my addr 172.21.15.129:0/926362286 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789dc75640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a6671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789e476640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a65c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 -- 172.21.15.129:0/926362286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 msgr2=0x55568a6671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a6671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 -- 172.21.15.129:0/926362286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 msgr2=0x55568a65c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a65c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789ec77640 1 -- 172.21.15.129:0/926362286 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568a66f010 con 0x55568a670240 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.300+0000 7f789e476640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a65c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f789ec77640 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7890003dc0 tx=0x7f7890016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476a893e2907b5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f7883fff640 1 -- 172.21.15.129:0/926362286 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890004b20 con 0x55568a670240 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f7883fff640 1 -- 172.21.15.129:0/926362286 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f789001f070 con 0x55568a670240 2024-09-05T01:23:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f7883fff640 1 -- 172.21.15.129:0/926362286 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890005020 con 0x55568a670240 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 -- 172.21.15.129:0/926362286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 msgr2=0x55568a687bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7890003dc0 tx=0x7f7890016000 comp rx=0 tx=0).stop 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 -- 172.21.15.129:0/926362286 shutdown_connections 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a670240 0x55568a687bd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665980 0x55568a6671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/926362286 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a665060 0x55568a65c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.301+0000 7f78a2ab3740 1 -- 172.21.15.129:0/926362286 >> 172.21.15.129:0/926362286 conn(0x55568a6615f0 msgr2=0x55568a666ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- 172.21.15.129:0/926362286 shutdown_connections 2024-09-05T01:23:15.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- 172.21.15.129:0/926362286 wait complete. 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- start start 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 Processor -- start 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a678540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a675330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00e350 con 0x55568a665980 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00f7c0 con 0x55568a670240 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f787c00ea20 con 0x55568a665060 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f78a2ab3740 10 client.0 _ll_register_callbacks cb 0x7f788c09b970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f789dc75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f789dc75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35658/0 (socket says 172.21.15.129:35658) 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f789dc75640 1 -- 172.21.15.129:0/2155822357 learned_addr learned my addr 172.21.15.129:0/2155822357 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:15.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f789e476640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a678540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.302+0000 7f789ec77640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a675330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 -- 172.21.15.129:0/2155822357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 msgr2=0x55568a678540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a678540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 -- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 msgr2=0x55568a675330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a675330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 -- 172.21.15.129:0/2155822357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568a6657b0 con 0x55568a665980 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789e476640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a678540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789ec77640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a675330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f789dc75640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c09ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9c7a6dc1e324d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c021070 con 0x55568a665980 2024-09-05T01:23:15.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f788c0a4700 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f789001f070 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f788c00ee60 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 10 client.16182 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55568a6eb480 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f789001f070 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f788c0b2040 con 0x55568a665980 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 10 client.16182 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 20 client.16182 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a6edb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 4 client.16182 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "26d479b5", "release": true, "await": true}] 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.303+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "26d479b5", "release": true, "await": true}) -- 0x55568a5d20a0 con 0x55568a67fec0 2024-09-05T01:23:15.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.304+0000 7f789e476640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a6edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:15.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.304+0000 7f789e476640 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a6edb00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7564da240ac93b11 server_cookie=6ee2e25ca76c7e39 in_seq=0 out_seq=0 2024-09-05T01:23:15.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.304+0000 7f7881ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:15.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.372+0000 7f7881ffb640 1 -- 172.21.15.129:0/2155822357 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55568a5d20a0 con 0x55568a67fec0 2024-09-05T01:23:15.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.372+0000 7f7881ffb640 10 client.16182 handle_command_reply: tid=0 2024-09-05T01:23:15.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.372+0000 7f78a2ab3740 1 client.16182 shutdown 2024-09-05T01:23:15.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 msgr2=0x55568a678a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f788c009fd0 tx=0x7f788c09ba90 comp rx=0 tx=0).stop 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 shutdown_connections 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55568a67fec0 0x55568a6edb00 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55568a670240 0x55568a675330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55568a665980 0x55568a678a80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 --2- 172.21.15.129:0/2155822357 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55568a665060 0x55568a678540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 >> 172.21.15.129:0/2155822357 conn(0x55568a6615f0 msgr2=0x55568a660750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f7881ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 shutdown_connections 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 1 -- 172.21.15.129:0/2155822357 wait complete. 2024-09-05T01:23:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.373+0000 7f78a2ab3740 20 client.16182 trim_cache size 0 max 16384 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 28, 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "26d479b5": { 2024-09-05T01:23:15.376 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 28, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 133.2, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:15.377 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:23:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.374+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 msgr2=0x7f78840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.374+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 0x7f78840a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f789806b430 tx=0x7f789000f040 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.374+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 msgr2=0x7f78981c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.374+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f788c00e370 tx=0x7f788c00e830 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 shutdown_connections 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f78840a2c00 0x7f78840a50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7898114ab0 0x7f78981c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7898109b10 0x7f78981c6bd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 --2- 172.21.15.129:0/184766535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f78981091f0 0x7f78981c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 >> 172.21.15.129:0/184766535 conn(0x7f789806e140 msgr2=0x7f7898071f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 shutdown_connections 2024-09-05T01:23:15.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:15.375+0000 7f78a06da640 1 -- 172.21.15.129:0/184766535 wait complete. 2024-09-05T01:23:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:15 smithi183 ceph-mon[34589]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2024-09-05T01:23:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:23:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:23:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:23:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:15 smithi194 ceph-mon[34941]: pgmap v589: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.8 MiB/s wr, 500 op/s 2024-09-05T01:23:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:23:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:23:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:23:16.179 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:16 smithi129 ceph-mon[30979]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-05T01:23:16.229 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '26d479b5', total seconds elapsed: 16.6 2024-09-05T01:23:16.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:16 smithi183 ceph-mon[34589]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-05T01:23:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:16 smithi194 ceph-mon[34941]: pgmap v590: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-09-05T01:23:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:19 smithi183 ceph-mon[34589]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-09-05T01:23:19.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:19 smithi129 ceph-mon[30979]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-09-05T01:23:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:19 smithi194 ceph-mon[34941]: pgmap v591: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.1 MiB/s wr, 221 op/s 2024-09-05T01:23:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 375.0 s, 1027.8 tps, lat 311.679 ms stddev 1545.377 2024-09-05T01:23:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:21 smithi129 ceph-mon[30979]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T01:23:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:21 smithi183 ceph-mon[34589]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T01:23:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:21 smithi194 ceph-mon[34941]: pgmap v592: 129 pgs: 129 active+clean; 1.3 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-05T01:23:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:23 smithi183 ceph-mon[34589]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-05T01:23:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:23 smithi129 ceph-mon[30979]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-05T01:23:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:23 smithi194 ceph-mon[34941]: pgmap v593: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-09-05T01:23:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 380.0 s, 835.2 tps, lat 119.302 ms stddev 102.615 2024-09-05T01:23:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:25 smithi183 ceph-mon[34589]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 468 op/s 2024-09-05T01:23:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:25 smithi129 ceph-mon[30979]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 468 op/s 2024-09-05T01:23:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:25 smithi194 ceph-mon[34941]: pgmap v594: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 4.6 MiB/s wr, 468 op/s 2024-09-05T01:23:26.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.108+0000 7ff84effd640 1 -- 172.21.15.129:0/4202380129 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838031880 con 0x7ff8501084c0 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 -- 172.21.15.129:0/4202380129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8501084c0 msgr2=0x7ff8501088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 --2- 172.21.15.129:0/4202380129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8501084c0 0x7ff8501088c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff838009930 tx=0x7ff83802a420 comp rx=0 tx=0).stop 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 -- 172.21.15.129:0/4202380129 shutdown_connections 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 --2- 172.21.15.129:0/4202380129 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff8501105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 --2- 172.21.15.129:0/4202380129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff850109380 0x7ff850109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 --2- 172.21.15.129:0/4202380129 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8501084c0 0x7ff8501088c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 -- 172.21.15.129:0/4202380129 >> 172.21.15.129:0/4202380129 conn(0x7ff850077490 msgr2=0x7ff8500798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.109+0000 7ff8565bd640 1 -- 172.21.15.129:0/4202380129 shutdown_connections 2024-09-05T01:23:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.110+0000 7ff8565bd640 1 -- 172.21.15.129:0/4202380129 wait complete. 2024-09-05T01:23:26.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.110+0000 7ff8565bd640 1 Processor -- start 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.110+0000 7ff8565bd640 1 -- start start 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8501084c0 0x7ff850080830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff850081db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff85007aef0 con 0x7ff8501084c0 2024-09-05T01:23:26.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff85007ad70 con 0x7ff850109380 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff8565bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff85007b070 con 0x7ff850109d40 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34992/0 (socket says 172.21.15.129:34992) 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff84f7fe640 1 -- 172.21.15.129:0/626606589 learned_addr learned my addr 172.21.15.129:0/626606589 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff84ffff640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8501084c0 0x7ff850080830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.111+0000 7ff854b5a640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff850081db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff854b5a640 1 -- 172.21.15.129:0/626606589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 msgr2=0x7ff850080d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff854b5a640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff854b5a640 1 -- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8501084c0 msgr2=0x7ff850080830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff854b5a640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8501084c0 0x7ff850080830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff854b5a640 1 -- 172.21.15.129:0/626606589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff850082420 con 0x7ff850109d40 2024-09-05T01:23:26.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.112+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:23:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff854b5a640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff850081db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff84800d680 tx=0x7ff84800db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff84d7fa640 1 -- 172.21.15.129:0/626606589 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848017070 con 0x7ff850109d40 2024-09-05T01:23:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff84d7fa640 1 -- 172.21.15.129:0/626606589 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff848009820 con 0x7ff850109d40 2024-09-05T01:23:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff84d7fa640 1 -- 172.21.15.129:0/626606589 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004cf0 con 0x7ff850109d40 2024-09-05T01:23:26.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff850082710 con 0x7ff850109d40 2024-09-05T01:23:26.117 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.114+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff850082c10 con 0x7ff850109d40 2024-09-05T01:23:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.117+0000 7ff84d7fa640 1 -- 172.21.15.129:0/626606589 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff8480099d0 con 0x7ff850109d40 2024-09-05T01:23:26.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.118+0000 7ff859432740 1 Processor -- start 2024-09-05T01:23:26.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.118+0000 7ff859432740 1 -- start start 2024-09-05T01:23:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff84d7fa640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 0x7ff83c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff84d7fa640 1 -- 172.21.15.129:0/626606589 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff848004a90 con 0x7ff850109d40 2024-09-05T01:23:26.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 0x7ff83c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff859432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff859432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b802e0 0x562611b84da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.119+0000 7ff859432740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b87830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff859432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562611ae32f0 con 0x562611b802e0 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff859432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562611a8b270 con 0x562611b7f9a0 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff859432740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562611adf2b0 con 0x562611b85370 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34998/0 (socket says 172.21.15.129:34998) 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 -- 172.21.15.129:0/2266942585 learned_addr learned my addr 172.21.15.129:0/2266942585 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:26.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff854b5a640 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b87830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 -- 172.21.15.129:0/2266942585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 msgr2=0x562611b87830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b87830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 -- 172.21.15.129:0/2266942585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b802e0 msgr2=0x562611b84da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b802e0 0x562611b84da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 -- 172.21.15.129:0/2266942585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562611a8a340 con 0x562611b7f9a0 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8380314e0 tx=0x7ff838005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7e25b8d3c868a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.124 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.120+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 0x7ff83c0a5350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff850081d40 tx=0x7ff84000ec60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.121+0000 7ff8467fc640 1 -- 172.21.15.129:0/2266942585 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838040070 con 0x562611b7f9a0 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.121+0000 7ff8467fc640 1 -- 172.21.15.129:0/2266942585 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff838003aa0 con 0x562611b7f9a0 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.121+0000 7ff8467fc640 1 -- 172.21.15.129:0/2266942585 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838035040 con 0x562611b7f9a0 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 -- 172.21.15.129:0/2266942585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 msgr2=0x562611b7fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff8380314e0 tx=0x7ff838005c00 comp rx=0 tx=0).stop 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 -- 172.21.15.129:0/2266942585 shutdown_connections 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b87830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b802e0 0x562611b84da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 --2- 172.21.15.129:0/2266942585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b7fda0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 -- 172.21.15.129:0/2266942585 >> 172.21.15.129:0/2266942585 conn(0x562611b7b010 msgr2=0x562611b7b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.122+0000 7ff859432740 1 -- 172.21.15.129:0/2266942585 shutdown_connections 2024-09-05T01:23:26.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.123+0000 7ff859432740 1 -- 172.21.15.129:0/2266942585 wait complete. 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.123+0000 7ff859432740 1 -- start start 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.123+0000 7ff859432740 1 Processor -- start 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b89c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562611b882e0 con 0x562611b85370 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562611a6c290 con 0x562611b7f9a0 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.124+0000 7ff859432740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562611b88460 con 0x562611b802e0 2024-09-05T01:23:26.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.125+0000 7ff859432740 10 client.0 _ll_register_callbacks cb 0x562611a9d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.125+0000 7ff854b5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.125+0000 7ff854b5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39750/0 (socket says 172.21.15.129:39750) 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.125+0000 7ff854b5a640 1 -- 172.21.15.129:0/2297271783 learned_addr learned my addr 172.21.15.129:0/2297271783 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.125+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 -- 172.21.15.129:0/2297271783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 msgr2=0x562611b8a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 -- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 msgr2=0x562611b89c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b89c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 -- 172.21.15.129:0/2297271783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626117f6210 con 0x562611b85370 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff854b5a640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff848097980 tx=0x7ff848009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcbd08f34a5d146 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.126+0000 7ff82f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848020040 con 0x562611b85370 2024-09-05T01:23:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff848004cd0 con 0x562611b85370 2024-09-05T01:23:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff838003aa0 con 0x562611b85370 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8480a3b70 con 0x562611b85370 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff859432740 10 client.16191 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.127+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5626118c2620 con 0x562611b85370 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff84809a400 con 0x562611b85370 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff848095c40 con 0x562611b85370 2024-09-05T01:23:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 10 client.16191 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 20 client.16191 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611b97c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 4 client.16191 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.128+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff838003aa0 con 0x562611b95840 2024-09-05T01:23:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.129+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611b97c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.132+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611b97c90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c01d9716acf43df server_cookie=341bfe69e3488164 in_seq=0 out_seq=0 2024-09-05T01:23:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.132+0000 7ff82f7fe640 10 client.16191 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.133+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2297271783 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff838003aa0 con 0x562611b95840 2024-09-05T01:23:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.133+0000 7ff82f7fe640 10 client.16191 handle_command_reply: tid=0 2024-09-05T01:23:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.133+0000 7ff859432740 1 client.16191 shutdown 2024-09-05T01:23:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.134+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 msgr2=0x562611b911a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.134+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff848097980 tx=0x7ff848009d60 comp rx=0 tx=0).stop 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.135+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 shutdown_connections 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611b97c90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b911a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff82f7fe640 0 client.16191 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff859432740 1 --2- 172.21.15.129:0/2297271783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b89c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.136+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 >> 172.21.15.129:0/2297271783 conn(0x562611b7b010 msgr2=0x562611b7c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.137+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 shutdown_connections 2024-09-05T01:23:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.137+0000 7ff859432740 1 -- 172.21.15.129:0/2297271783 wait complete. 2024-09-05T01:23:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.137+0000 7ff859432740 20 client.16191 trim_cache size 0 max 16384 2024-09-05T01:23:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.172+0000 7ff859432740 1 Processor -- start 2024-09-05T01:23:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.172+0000 7ff859432740 1 -- start start 2024-09-05T01:23:26.175 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b7f9a0 0x562611b7a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b9c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501bc040 con 0x562611b7f9a0 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501b9910 con 0x562611b802e0 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff859432740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501b7760 con 0x562611b85370 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35016/0 (socket says 172.21.15.129:35016) 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 -- 172.21.15.129:0/185039911 learned_addr learned my addr 172.21.15.129:0/185039911 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff854b5a640 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b9c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 -- 172.21.15.129:0/185039911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 msgr2=0x562611b9c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b9c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 -- 172.21.15.129:0/185039911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b7f9a0 msgr2=0x562611b7a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b7f9a0 0x562611b7a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.173+0000 7ff84f7fe640 1 -- 172.21.15.129:0/185039911 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562611b807c0 con 0x562611b802e0 2024-09-05T01:23:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.174+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8400107a0 tx=0x7ff840015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79b306b5ca5790a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.179 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.177+0000 7ff845ffb640 1 -- 172.21.15.129:0/185039911 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840021430 con 0x562611b802e0 2024-09-05T01:23:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.178+0000 7ff845ffb640 1 -- 172.21.15.129:0/185039911 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff840017040 con 0x562611b802e0 2024-09-05T01:23:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.178+0000 7ff845ffb640 1 -- 172.21.15.129:0/185039911 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840020730 con 0x562611b802e0 2024-09-05T01:23:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 -- 172.21.15.129:0/185039911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 msgr2=0x562611b91160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8400107a0 tx=0x7ff840015000 comp rx=0 tx=0).stop 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 -- 172.21.15.129:0/185039911 shutdown_connections 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b85370 0x562611b9c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b802e0 0x562611b91160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 --2- 172.21.15.129:0/185039911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b7f9a0 0x562611b7a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.179+0000 7ff859432740 1 -- 172.21.15.129:0/185039911 >> 172.21.15.129:0/185039911 conn(0x562611b7b010 msgr2=0x562611b799a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.180+0000 7ff859432740 1 -- 172.21.15.129:0/185039911 shutdown_connections 2024-09-05T01:23:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.182+0000 7ff859432740 1 -- 172.21.15.129:0/185039911 wait complete. 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.182+0000 7ff859432740 1 -- start start 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 Processor -- start 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501b7dd0 con 0x562611b85370 2024-09-05T01:23:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501b9240 con 0x562611b7f9a0 2024-09-05T01:23:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501b84a0 con 0x562611b802e0 2024-09-05T01:23:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.183+0000 7ff859432740 10 client.0 _ll_register_callbacks cb 0x7ff8480bb0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.185+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.185+0000 7ff84f7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48196/0 (socket says 172.21.15.129:48196) 2024-09-05T01:23:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.185+0000 7ff84f7fe640 1 -- 172.21.15.129:0/2116941408 learned_addr learned my addr 172.21.15.129:0/2116941408 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.185+0000 7ff854b5a640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.185+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 -- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 msgr2=0x562611b8f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b8f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 -- 172.21.15.129:0/2116941408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 msgr2=0x562611b90790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b90790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 -- 172.21.15.129:0/2116941408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562611b90da0 con 0x562611b802e0 2024-09-05T01:23:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff84f7fe640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff84000a010 tx=0x7ff840021640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=533458e0569dee57 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff82f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:23:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840038060 con 0x562611b802e0 2024-09-05T01:23:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.186+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7ff840021970 con 0x562611b802e0 2024-09-05T01:23:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.187+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff840017040 con 0x562611b802e0 2024-09-05T01:23:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.187+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff840033470 con 0x562611b802e0 2024-09-05T01:23:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.188+0000 7ff859432740 10 client.25693 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.188+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562611c11f50 con 0x562611b802e0 2024-09-05T01:23:26.190 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.188+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff840033ac0 con 0x562611b802e0 2024-09-05T01:23:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.189+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7ff840020470 con 0x562611b802e0 2024-09-05T01:23:26.191 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.189+0000 7ff859432740 10 client.25693 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.190+0000 7ff859432740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.190+0000 7ff859432740 20 client.25693 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.190+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611c14880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.190+0000 7ff859432740 4 client.25693 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:23:26.192 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.190+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ff840017040 con 0x562611b95840 2024-09-05T01:23:26.193 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.191+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611c14880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:26.196 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.194+0000 7ff84ffff640 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611c14880 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a9e1459b62598c server_cookie=208751e626070326 in_seq=0 out_seq=0 2024-09-05T01:23:26.197 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.194+0000 7ff82f7fe640 10 client.25693 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.213+0000 7ff82f7fe640 1 -- 172.21.15.129:0/2116941408 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7ff840017040 con 0x562611b95840 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.213+0000 7ff82f7fe640 10 client.25693 handle_command_reply: tid=0 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.213+0000 7ff859432740 1 client.25693 shutdown 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 msgr2=0x562611b8f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff84000a010 tx=0x7ff840021640 comp rx=0 tx=0).stop 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 shutdown_connections 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562611b95840 0x562611c14880 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff82f7fe640 0 client.25693 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562611b85370 0x562611b90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562611b802e0 0x562611b8f780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 --2- 172.21.15.129:0/2116941408 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562611b7f9a0 0x562611b8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 >> 172.21.15.129:0/2116941408 conn(0x562611b7b010 msgr2=0x562611b7a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 shutdown_connections 2024-09-05T01:23:26.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 1 -- 172.21.15.129:0/2116941408 wait complete. 2024-09-05T01:23:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.214+0000 7ff859432740 20 client.25693 trim_cache size 0 max 16384 2024-09-05T01:23:26.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:23:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:23:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:23:26.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.217+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 msgr2=0x7ff83c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.217+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 0x7ff83c0a5350 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff850081d40 tx=0x7ff84000ec60 comp rx=0 tx=0).stop 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.217+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 msgr2=0x7ff850081db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.217+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff850081db0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff84800d680 tx=0x7ff84800db40 comp rx=0 tx=0).stop 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 shutdown_connections 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff83c0a2e90 0x7ff83c0a5350 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff850109d40 0x7ff850081db0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff850109380 0x7ff850080d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 --2- 172.21.15.129:0/626606589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff8501084c0 0x7ff850080830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 >> 172.21.15.129:0/626606589 conn(0x7ff850077490 msgr2=0x7ff850079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 shutdown_connections 2024-09-05T01:23:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:26.218+0000 7ff8565bd640 1 -- 172.21.15.129:0/626606589 wait complete. 2024-09-05T01:23:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:27 smithi183 ceph-mon[34589]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 576 op/s 2024-09-05T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:27 smithi129 ceph-mon[30979]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 576 op/s 2024-09-05T01:23:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:27 smithi194 ceph-mon[34941]: pgmap v595: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.0 MiB/s wr, 576 op/s 2024-09-05T01:23:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:29 smithi183 ceph-mon[34589]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 765 op/s 2024-09-05T01:23:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:29 smithi129 ceph-mon[30979]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 765 op/s 2024-09-05T01:23:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:29 smithi194 ceph-mon[34941]: pgmap v596: 129 pgs: 129 active+clean; 1.3 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 765 op/s 2024-09-05T01:23:30.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 385.0 s, 958.1 tps, lat 104.851 ms stddev 52.360 2024-09-05T01:23:31.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:31 smithi183 ceph-mon[34589]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 814 op/s 2024-09-05T01:23:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:31 smithi129 ceph-mon[30979]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 814 op/s 2024-09-05T01:23:31.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:31 smithi194 ceph-mon[34941]: pgmap v597: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.3 MiB/s wr, 814 op/s 2024-09-05T01:23:33.301 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:33 smithi194 ceph-mon[34941]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:33.325 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:33 smithi129 ceph-mon[30979]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:33.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:33 smithi183 ceph-mon[34589]: pgmap v598: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 786 op/s 2024-09-05T01:23:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 390.0 s, 986.1 tps, lat 101.388 ms stddev 33.372 2024-09-05T01:23:35.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:35 smithi183 ceph-mon[34589]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 969 op/s 2024-09-05T01:23:35.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:35 smithi129 ceph-mon[30979]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 969 op/s 2024-09-05T01:23:35.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:35 smithi194 ceph-mon[34941]: pgmap v599: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.4 MiB/s wr, 969 op/s 2024-09-05T01:23:37.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:37 smithi183 ceph-mon[34589]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.6 MiB/s wr, 870 op/s 2024-09-05T01:23:37.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:37 smithi129 ceph-mon[30979]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.6 MiB/s wr, 870 op/s 2024-09-05T01:23:37.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:37 smithi194 ceph-mon[34941]: pgmap v600: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.6 MiB/s wr, 870 op/s 2024-09-05T01:23:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbaf7fe640 1 -- 172.21.15.129:0/1960577327 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8031e50 con 0x7fdbb01025f0 2024-09-05T01:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 -- 172.21.15.129:0/1960577327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb01025f0 msgr2=0x7fdbb00698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/1960577327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb01025f0 0x7fdbb00698a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009ca0 tx=0x7fdba802a8d0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 -- 172.21.15.129:0/1960577327 shutdown_connections 2024-09-05T01:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/1960577327 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb01025f0 0x7fdbb00698a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/1960577327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb0101c30 0x7fdbb01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/1960577327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.372+0000 7fdbb68b7640 1 -- 172.21.15.129:0/1960577327 >> 172.21.15.129:0/1960577327 conn(0x7fdbb00fc1e0 msgr2=0x7fdbb00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.374+0000 7fdbb68b7640 1 -- 172.21.15.129:0/1960577327 shutdown_connections 2024-09-05T01:23:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.374+0000 7fdbb68b7640 1 -- 172.21.15.129:0/1960577327 wait complete. 2024-09-05T01:23:39.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.374+0000 7fdbb68b7640 1 Processor -- start 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 -- start start 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb0101c30 0x7fdbb019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb01025f0 0x7fdbb01a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010f950 con 0x7fdbb0100a40 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010f7d0 con 0x7fdbb01025f0 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbb68b7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb010fad0 con 0x7fdbb0101c30 2024-09-05T01:23:39.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55048/0 (socket says 172.21.15.129:55048) 2024-09-05T01:23:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.375+0000 7fdbaffff640 1 -- 172.21.15.129:0/2519407753 learned_addr learned my addr 172.21.15.129:0/2519407753 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb01025f0 0x7fdbb01a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/2519407753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb0101c30 msgr2=0x7fdbb019e880 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:23:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb0101c30 0x7fdbb019e880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/2519407753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb01025f0 msgr2=0x7fdbb01a58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb01025f0 0x7fdbb01a58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 -- 172.21.15.129:0/2519407753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb01a5f60 con 0x7fdbb0100a40 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.376+0000 7fdbaffff640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d770 tx=0x7fdba000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.377+0000 7fdbadffb640 1 -- 172.21.15.129:0/2519407753 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0017070 con 0x7fdbb0100a40 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.377+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb01a6250 con 0x7fdbb0100a40 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.377+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb01a6870 con 0x7fdbb0100a40 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.377+0000 7fdbadffb640 1 -- 172.21.15.129:0/2519407753 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdba0009c30 con 0x7fdbb0100a40 2024-09-05T01:23:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.377+0000 7fdbadffb640 1 -- 172.21.15.129:0/2519407753 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003d00 con 0x7fdbb0100a40 2024-09-05T01:23:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.382+0000 7fdbadffb640 1 -- 172.21.15.129:0/2519407753 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdba0003ea0 con 0x7fdbb0100a40 2024-09-05T01:23:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 Processor -- start 2024-09-05T01:23:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 -- start start 2024-09-05T01:23:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8772192f0 con 0x55d8772b62e0 2024-09-05T01:23:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8771c1270 con 0x55d8772b59a0 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.383+0000 7fdbb9653740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d8772152b0 con 0x55d8772bb370 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdbb4e54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdbadffb640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 0x7fdb9c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdbadffb640 1 -- 172.21.15.129:0/2519407753 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdba00ced50 con 0x7fdbb0100a40 2024-09-05T01:23:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdbb4e54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:33886/0 (socket says 172.21.15.129:33886) 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.384+0000 7fdbb4e54640 1 -- 172.21.15.129:0/713871358 learned_addr learned my addr 172.21.15.129:0/713871358 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 0x7fdb9c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbaffff640 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 -- 172.21.15.129:0/713871358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 msgr2=0x55d8772bd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772bd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 -- 172.21.15.129:0/713871358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 msgr2=0x55d8772bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbb4e54640 1 -- 172.21.15.129:0/713871358 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8771c0340 con 0x55d8772b59a0 2024-09-05T01:23:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.385+0000 7fdbaffff640 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.386+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdba802aec0 tx=0x7fdba8031ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d57cec84a3db5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.388+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 0x7fdb9c0a51f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdba8048000 tx=0x7fdba804a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.393+0000 7fdba67fc640 1 -- 172.21.15.129:0/713871358 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8031620 con 0x55d8772b59a0 2024-09-05T01:23:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.393+0000 7fdba67fc640 1 -- 172.21.15.129:0/713871358 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdba8036070 con 0x55d8772b59a0 2024-09-05T01:23:39.394 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.394+0000 7fdba67fc640 1 -- 172.21.15.129:0/713871358 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8030900 con 0x55d8772b59a0 2024-09-05T01:23:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 -- 172.21.15.129:0/713871358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 msgr2=0x55d8772b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.396 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdba802aec0 tx=0x7fdba8031ef0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 -- 172.21.15.129:0/713871358 shutdown_connections 2024-09-05T01:23:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772bd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 --2- 172.21.15.129:0/713871358 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772b5da0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.395+0000 7fdbb9653740 1 -- 172.21.15.129:0/713871358 >> 172.21.15.129:0/713871358 conn(0x55d8772b1010 msgr2=0x55d8772b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.397+0000 7fdbb9653740 1 -- 172.21.15.129:0/713871358 shutdown_connections 2024-09-05T01:23:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.397+0000 7fdbb9653740 1 -- 172.21.15.129:0/713871358 wait complete. 2024-09-05T01:23:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.397+0000 7fdbb9653740 1 -- start start 2024-09-05T01:23:39.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 Processor -- start 2024-09-05T01:23:39.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d8772be280 con 0x55d8772b62e0 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.398+0000 7fdbb9653740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d8771a2290 con 0x55d8772b59a0 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb9653740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d8772be400 con 0x55d8772bb370 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb9653740 10 client.0 _ll_register_callbacks cb 0x55d8771d3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.400 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:59506/0 (socket says 172.21.15.129:59506) 2024-09-05T01:23:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 -- 172.21.15.129:0/3729247969 learned_addr learned my addr 172.21.15.129:0/3729247969 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:39.401 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbaffff640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 -- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 msgr2=0x55d8772bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 -- 172.21.15.129:0/3729247969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 msgr2=0x55d8772c0170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:23:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.402 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.399+0000 7fdbb4e54640 1 -- 172.21.15.129:0/3729247969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d876f2c210 con 0x55d8772bb370 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.400+0000 7fdbb4e54640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009430 tx=0x7fdba802a8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e602f6de0645a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.400+0000 7fdb8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.400+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8057030 con 0x55d8772bb370 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.400+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdba8003e00 con 0x55d8772bb370 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.400+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba8036070 con 0x55d8772bb370 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.401+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdba8004b40 con 0x55d8772bb370 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.401+0000 7fdbb9653740 10 client.25702 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:39.403 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.401+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d876ff8620 con 0x55d8772bb370 2024-09-05T01:23:39.404 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.404+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdba8036070 con 0x55d8772bb370 2024-09-05T01:23:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdba805e4d0 con 0x55d8772bb370 2024-09-05T01:23:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 10 client.25702 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:39.405 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 20 client.25702 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d8772dc800 0x55d8772deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 4 client.25702 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:39.406 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d877226a20 con 0x55d8772dc800 2024-09-05T01:23:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.405+0000 7fdbaffff640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d8772dc800 0x55d8772deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.406+0000 7fdbaffff640 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d8772dc800 0x55d8772deca0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fffb16e17b537eb server_cookie=86a5fe50dd210a6d in_seq=0 out_seq=0 2024-09-05T01:23:39.407 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.406+0000 7fdb8f7fe640 10 client.25702 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/3729247969 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d877226a20 con 0x55d8772dc800 2024-09-05T01:23:39.408 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdb8f7fe640 10 client.25702 handle_command_reply: tid=0 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 client.25702 shutdown 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 msgr2=0x55d8772c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009430 tx=0x7fdba802a8d0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 shutdown_connections 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d8772dc800 0x55d8772deca0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772bb370 0x55d8772c70d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3729247969 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.407+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 >> 172.21.15.129:0/3729247969 conn(0x55d8772b1010 msgr2=0x55d8772b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.409 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.408+0000 7fdb8f7fe640 0 client.25702 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.408+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 shutdown_connections 2024-09-05T01:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.408+0000 7fdbb9653740 1 -- 172.21.15.129:0/3729247969 wait complete. 2024-09-05T01:23:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.409+0000 7fdbb9653740 20 client.25702 trim_cache size 0 max 16384 2024-09-05T01:23:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:39 smithi183 ceph-mon[34589]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 976 op/s 2024-09-05T01:23:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 Processor -- start 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 -- start start 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772bb370 0x55d8772c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90012760 con 0x55d8772bb370 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90010030 con 0x55d8772b59a0 2024-09-05T01:23:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.443+0000 7fdbb9653740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000de80 con 0x55d8772b62e0 2024-09-05T01:23:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:59516/0 (socket says 172.21.15.129:59516) 2024-09-05T01:23:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 -- 172.21.15.129:0/3543793933 learned_addr learned my addr 172.21.15.129:0/3543793933 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdbaffff640 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 -- 172.21.15.129:0/3543793933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 msgr2=0x55d8772bfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 -- 172.21.15.129:0/3543793933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772bb370 msgr2=0x55d8772c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772bb370 0x55d8772c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.444+0000 7fdba77fe640 1 -- 172.21.15.129:0/3543793933 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8772bf770 con 0x55d8772b62e0 2024-09-05T01:23:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.445+0000 7fdbaffff640 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.445+0000 7fdba77fe640 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb980096b0 tx=0x7fdb98009b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d730ab5c1de221 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.445+0000 7fdba5ffb640 1 -- 172.21.15.129:0/3543793933 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98039070 con 0x55d8772b62e0 2024-09-05T01:23:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.445+0000 7fdba5ffb640 1 -- 172.21.15.129:0/3543793933 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb98030bf0 con 0x55d8772b62e0 2024-09-05T01:23:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.445+0000 7fdba5ffb640 1 -- 172.21.15.129:0/3543793933 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035dc0 con 0x55d8772b62e0 2024-09-05T01:23:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.446+0000 7fdbb9653740 1 -- 172.21.15.129:0/3543793933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 msgr2=0x55d8772c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.446+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdb980096b0 tx=0x7fdb98009b70 comp rx=0 tx=0).stop 2024-09-05T01:23:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 -- 172.21.15.129:0/3543793933 shutdown_connections 2024-09-05T01:23:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772bb370 0x55d8772c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b62e0 0x55d8772c7090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 --2- 172.21.15.129:0/3543793933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772b59a0 0x55d8772bfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 -- 172.21.15.129:0/3543793933 >> 172.21.15.129:0/3543793933 conn(0x55d8772b1010 msgr2=0x55d8772b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 -- 172.21.15.129:0/3543793933 shutdown_connections 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 -- 172.21.15.129:0/3543793933 wait complete. 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.447+0000 7fdbb9653740 1 -- start start 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 Processor -- start 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b59a0 0x55d8772c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772bb370 0x55d877345530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000e4f0 con 0x55d8772b62e0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000f960 con 0x55d8772bb370 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9000ebc0 con 0x55d8772b59a0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.448+0000 7fdbb9653740 10 client.0 _ll_register_callbacks cb 0x7fdb98006e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.449+0000 7fdbb4e54640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772bb370 0x55d877345530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.449+0000 7fdbaffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b59a0 0x55d8772c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.449+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.449+0000 7fdba77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55086/0 (socket says 172.21.15.129:55086) 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.449+0000 7fdba77fe640 1 -- 172.21.15.129:0/1204599053 learned_addr learned my addr 172.21.15.129:0/1204599053 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 -- 172.21.15.129:0/1204599053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b59a0 msgr2=0x55d8772c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b59a0 0x55d8772c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 -- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772bb370 msgr2=0x55d877345530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772bb370 0x55d877345530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 -- 172.21.15.129:0/1204599053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8772c7300 con 0x55d8772b62e0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdba77fe640 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb9803b000 tx=0x7fdb98035780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c80f5fa9b50a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.450+0000 7fdb8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98039050 con 0x55d8772b62e0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdb98035cc0 con 0x55d8772b62e0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb98030bf0 con 0x55d8772b62e0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb98035480 con 0x55d8772b62e0 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdbb9653740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.451+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d877346a70 con 0x55d8772b62e0 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb98030bf0 con 0x55d8772b62e0 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdb98048510 con 0x55d8772b62e0 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 20 client.16206 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d877347370 0x55d877349790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 4 client.16206 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.452+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d877220d70 con 0x55d877347370 2024-09-05T01:23:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.453+0000 7fdbaffff640 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d877347370 0x55d877349790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.454+0000 7fdbaffff640 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d877347370 0x55d877349790 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797aaaec62ee32b7 server_cookie=1d5ebb17f71fbcb7 in_seq=0 out_seq=0 2024-09-05T01:23:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.454+0000 7fdb8f7fe640 10 client.16206 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdb8f7fe640 1 -- 172.21.15.129:0/1204599053 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d877220d70 con 0x55d877347370 2024-09-05T01:23:39.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdb8f7fe640 10 client.16206 handle_command_reply: tid=0 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 client.16206 shutdown 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 msgr2=0x55d8772c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdb9803b000 tx=0x7fdb98035780 comp rx=0 tx=0).stop 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 shutdown_connections 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d877347370 0x55d877349790 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d8772bb370 0x55d877345530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d8772b62e0 0x55d8772c4210 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 --2- 172.21.15.129:0/1204599053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d8772b59a0 0x55d8772c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 >> 172.21.15.129:0/1204599053 conn(0x55d8772b1010 msgr2=0x55d8772b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdb8f7fe640 0 client.16206 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:39.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 shutdown_connections 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.488+0000 7fdbb9653740 1 -- 172.21.15.129:0/1204599053 wait complete. 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.489+0000 7fdbb9653740 20 client.16206 trim_cache size 0 max 16384 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:23:39.491 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:23:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.489+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 msgr2=0x7fdb9c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.489+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 0x7fdb9c0a51f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdba8048000 tx=0x7fdba804a000 comp rx=0 tx=0).stop 2024-09-05T01:23:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 msgr2=0x7fdbb019e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d770 tx=0x7fdba000dc30 comp rx=0 tx=0).stop 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 shutdown_connections 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb9c0a2d30 0x7fdb9c0a51f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdbb01025f0 0x7fdbb01a58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdbb0101c30 0x7fdbb019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 --2- 172.21.15.129:0/2519407753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbb0100a40 0x7fdbb019e340 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.490+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 >> 172.21.15.129:0/2519407753 conn(0x7fdbb00fc1e0 msgr2=0x7fdbb0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.491+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 shutdown_connections 2024-09-05T01:23:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:39.491+0000 7fdbb68b7640 1 -- 172.21.15.129:0/2519407753 wait complete. 2024-09-05T01:23:39.495 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:39 smithi129 ceph-mon[30979]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 976 op/s 2024-09-05T01:23:39.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:39 smithi194 ceph-mon[34941]: pgmap v601: 129 pgs: 129 active+clean; 1.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 976 op/s 2024-09-05T01:23:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 395.0 s, 928.6 tps, lat 107.902 ms stddev 49.898 2024-09-05T01:23:40.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:40 smithi183 ceph-mon[34589]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-09-05T01:23:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:40 smithi129 ceph-mon[30979]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-09-05T01:23:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:40 smithi194 ceph-mon[34941]: pgmap v602: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 9.0 MiB/s wr, 903 op/s 2024-09-05T01:23:40.745 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T01:23:40.745 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T01:23:40.746 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:23:41.272 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:43.071 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:43 smithi129 ceph-mon[30979]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 -- 172.21.15.129:0/973249037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 msgr2=0x7f17a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 --2- 172.21.15.129:0/973249037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a810da40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f17a4009890 tx=0x7f17a402a3f0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 -- 172.21.15.129:0/973249037 shutdown_connections 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 --2- 172.21.15.129:0/973249037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a810da40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 --2- 172.21.15.129:0/973249037 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 0x7f17a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 --2- 172.21.15.129:0/973249037 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 0x7f17a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.206+0000 7f17af456640 1 -- 172.21.15.129:0/973249037 >> 172.21.15.129:0/973249037 conn(0x7f17a80fc7d0 msgr2=0x7f17a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:43.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.207+0000 7f17af456640 1 -- 172.21.15.129:0/973249037 shutdown_connections 2024-09-05T01:23:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.207+0000 7f17af456640 1 -- 172.21.15.129:0/973249037 wait complete. 2024-09-05T01:23:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 Processor -- start 2024-09-05T01:23:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 -- start start 2024-09-05T01:23:43.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 0x7f17a81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 0x7f17a81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810fd30 con 0x7f17a8100e20 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810fbb0 con 0x7f17a8102ef0 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.208+0000 7f17af456640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f17a810feb0 con 0x7f17a8102570 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34318/0 (socket says 172.21.15.129:34318) 2024-09-05T01:23:43.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 -- 172.21.15.129:0/4105075742 learned_addr learned my addr 172.21.15.129:0/4105075742 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ac9f1640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 0x7f17a81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad1f2640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 0x7f17a81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 -- 172.21.15.129:0/4105075742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 msgr2=0x7f17a81a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 0x7f17a81a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 -- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 msgr2=0x7f17a81a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 0x7f17a81a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 -- 172.21.15.129:0/4105075742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a819eff0 con 0x7f17a8102ef0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.209+0000 7f17ad9f3640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17a4035040 tx=0x7f17a402ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.210+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4037040 con 0x7f17a8102ef0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.210+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17a4041070 con 0x7f17a8102ef0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.210+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4037040 con 0x7f17a8102ef0 2024-09-05T01:23:43.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.210+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a819f280 con 0x7f17a8102ef0 2024-09-05T01:23:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.210+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a819f6b0 con 0x7f17a8102ef0 2024-09-05T01:23:43.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.212+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a81021b0 con 0x7f17a8102ef0 2024-09-05T01:23:43.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.213+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f17a40035f0 con 0x7f17a8102ef0 2024-09-05T01:23:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.214+0000 7f179e7fc640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 0x7f17940a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:43.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.215+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f17a40eee70 con 0x7f17a8102ef0 2024-09-05T01:23:43.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.215+0000 7f17ad1f2640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 0x7f17940a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:43.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.216+0000 7f17ad1f2640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 0x7f17940a5280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005fd0 tx=0x7f17a0010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:43.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.218+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17a40bb9c0 con 0x7f17a8102ef0 2024-09-05T01:23:43.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:43 smithi183 ceph-mon[34589]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-09-05T01:23:43.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:43 smithi194 ceph-mon[34941]: pgmap v603: 129 pgs: 129 active+clean; 1.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 7.7 MiB/s wr, 822 op/s 2024-09-05T01:23:43.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.618+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a81090c0 con 0x7f17a8102ef0 2024-09-05T01:23:43.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.619+0000 7f179e7fc640 1 -- 172.21.15.129:0/4105075742 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f17a40bb2c0 con 0x7f17a8102ef0 2024-09-05T01:23:43.621 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:23:43.621 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:23:43.621 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.623+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 msgr2=0x7f17940a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.623+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 0x7f17940a5280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005fd0 tx=0x7f17a0010040 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.623+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 msgr2=0x7f17a819e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.623+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f17a4035040 tx=0x7f17a402ae20 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 shutdown_connections 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f17940a2dc0 0x7f17940a5280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f17a8102ef0 0x7f17a819e850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f17a8102570 0x7f17a81a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 --2- 172.21.15.129:0/4105075742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f17a8100e20 0x7f17a81a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:43.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.624+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 >> 172.21.15.129:0/4105075742 conn(0x7f17a80fc7d0 msgr2=0x7f17a81048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:43.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.625+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 shutdown_connections 2024-09-05T01:23:43.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:43.625+0000 7f17af456640 1 -- 172.21.15.129:0/4105075742 wait complete. 2024-09-05T01:23:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:44 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/4105075742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:44.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:44 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/4105075742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:44 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/4105075742' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:23:44.623 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T01:23:44.624 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T01:23:45.083 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:23:45.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 400.0 s, 799.6 tps, lat 124.849 ms stddev 113.626 2024-09-05T01:23:45.232 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:45 smithi129 ceph-mon[30979]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-05T01:23:45.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:45 smithi183 ceph-mon[34589]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-05T01:23:45.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:45 smithi194 ceph-mon[34941]: pgmap v604: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.5 MiB/s wr, 1.01k op/s 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 -- 172.21.15.129:0/245548952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c105f40 msgr2=0x7fdb5c1063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 --2- 172.21.15.129:0/245548952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c105f40 0x7fdb5c1063d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb5001a910 tx=0x7fdb5003b4f0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 -- 172.21.15.129:0/245548952 shutdown_connections 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 --2- 172.21.15.129:0/245548952 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c105f40 0x7fdb5c1063d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 --2- 172.21.15.129:0/245548952 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 --2- 172.21.15.129:0/245548952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c100b30 0x7fdb5c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 -- 172.21.15.129:0/245548952 >> 172.21.15.129:0/245548952 conn(0x7fdb5c0fc7a0 msgr2=0x7fdb5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:45.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.802+0000 7fdb620a5640 1 -- 172.21.15.129:0/245548952 shutdown_connections 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.803+0000 7fdb620a5640 1 -- 172.21.15.129:0/245548952 wait complete. 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.803+0000 7fdb620a5640 1 Processor -- start 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 -- start start 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 0x7fdb5c114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c1151f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c105f40 0x7fdb5c110f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c109730 con 0x7fdb5c105f40 2024-09-05T01:23:45.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1095b0 con 0x7fdb5c101450 2024-09-05T01:23:45.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.804+0000 7fdb620a5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1098b0 con 0x7fdb5c100b30 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.805+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c1151f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.805+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c1151f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34336/0 (socket says 172.21.15.129:34336) 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.805+0000 7fdb5affd640 1 -- 172.21.15.129:0/3176991838 learned_addr learned my addr 172.21.15.129:0/3176991838 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.805+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 0x7fdb5c114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.805+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c105f40 0x7fdb5c110f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 -- 172.21.15.129:0/3176991838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 msgr2=0x7fdb5c1151f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c1151f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 -- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c105f40 msgr2=0x7fdb5c110f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c105f40 0x7fdb5c110f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.807 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 -- 172.21.15.129:0/3176991838 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c111760 con 0x7fdb5c100b30 2024-09-05T01:23:45.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 0x7fdb5c114cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400dae0 tx=0x7fdb5400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.808 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb58ff9640 1 -- 172.21.15.129:0/3176991838 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017030 con 0x7fdb5c100b30 2024-09-05T01:23:45.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb58ff9640 1 -- 172.21.15.129:0/3176991838 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb54003e20 con 0x7fdb5c100b30 2024-09-05T01:23:45.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.806+0000 7fdb58ff9640 1 -- 172.21.15.129:0/3176991838 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54010460 con 0x7fdb5c100b30 2024-09-05T01:23:45.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.807+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c111a50 con 0x7fdb5c100b30 2024-09-05T01:23:45.809 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.808+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c06bde0 con 0x7fdb5c100b30 2024-09-05T01:23:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.812+0000 7fdb6447e740 1 Processor -- start 2024-09-05T01:23:45.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.812+0000 7fdb6447e740 1 -- start start 2024-09-05T01:23:45.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463c080 0x55c304646400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304648d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c3045a12a0 con 0x55c304646940 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c304548b10 con 0x55c30463b760 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.813+0000 7fdb6447e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c30459d260 con 0x55c30463c080 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34350/0 (socket says 172.21.15.129:34350) 2024-09-05T01:23:45.815 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 -- 172.21.15.129:0/3982006482 learned_addr learned my addr 172.21.15.129:0/3982006482 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5affd640 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304648d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 -- 172.21.15.129:0/3982006482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463c080 msgr2=0x55c304646400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463c080 0x55c304646400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 -- 172.21.15.129:0/3982006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 msgr2=0x55c304648d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304648d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5bfff640 1 -- 172.21.15.129:0/3982006482 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c304312b90 con 0x55c30463b760 2024-09-05T01:23:45.816 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.814+0000 7fdb5affd640 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304648d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:45.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.815+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb50043000 tx=0x7fdb50004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b1dc9ae74ed9da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.815+0000 7fdb41ffb640 1 -- 172.21.15.129:0/3982006482 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50004040 con 0x55c30463b760 2024-09-05T01:23:45.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.815+0000 7fdb41ffb640 1 -- 172.21.15.129:0/3982006482 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb500041e0 con 0x55c30463b760 2024-09-05T01:23:45.817 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.815+0000 7fdb41ffb640 1 -- 172.21.15.129:0/3982006482 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50040880 con 0x55c30463b760 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.816+0000 7fdb6447e740 1 -- 172.21.15.129:0/3982006482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 msgr2=0x55c30463bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.816+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb50043000 tx=0x7fdb50004770 comp rx=0 tx=0).stop 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 -- 172.21.15.129:0/3982006482 shutdown_connections 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304648d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463c080 0x55c304646400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3982006482 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c30463b760 0x55c30463bb40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 -- 172.21.15.129:0/3982006482 >> 172.21.15.129:0/3982006482 conn(0x55c304637cf0 msgr2=0x55c3046380d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 -- 172.21.15.129:0/3982006482 shutdown_connections 2024-09-05T01:23:45.818 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 -- 172.21.15.129:0/3982006482 wait complete. 2024-09-05T01:23:45.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb6447e740 1 -- start start 2024-09-05T01:23:45.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.817+0000 7fdb58ff9640 1 -- 172.21.15.129:0/3176991838 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdb5401a070 con 0x7fdb5c100b30 2024-09-05T01:23:45.819 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 Processor -- start 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c304643950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304643e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c304649510 con 0x55c304646940 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c3045296e0 con 0x55c3046443d0 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c304649690 con 0x55c30463b760 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.818+0000 7fdb6447e740 10 client.0 _ll_register_callbacks cb 0x55c30454da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.819+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.819+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34356/0 (socket says 172.21.15.129:34356) 2024-09-05T01:23:45.821 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.819+0000 7fdb5affd640 1 -- 172.21.15.129:0/1127608287 learned_addr learned my addr 172.21.15.129:0/1127608287 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:45.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.819+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304643e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.820+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c304643950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.822 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.820+0000 7fdb5affd640 1 -- 172.21.15.129:0/1127608287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 msgr2=0x55c304643950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.821+0000 7fdb58ff9640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 0x7fdb440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.821+0000 7fdb58ff9640 1 -- 172.21.15.129:0/3176991838 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdb540ce2b0 con 0x7fdb5c100b30 2024-09-05T01:23:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c304643950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5affd640 1 -- 172.21.15.129:0/1127608287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 msgr2=0x55c304643e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.826 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304643e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5affd640 1 -- 172.21.15.129:0/1127608287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c304652f70 con 0x55c3046443d0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 0x7fdb440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.822+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0126d0 tx=0x7fdb4c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70118d47609ba6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb3affd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f9a0 con 0x55c3046443d0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb500041e0 con 0x55c3046443d0 2024-09-05T01:23:45.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdb4c00fb40 con 0x55c3046443d0 2024-09-05T01:23:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c00fd40 con 0x55c3046443d0 2024-09-05T01:23:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb6447e740 10 client.25769 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c304655940 con 0x55c3046443d0 2024-09-05T01:23:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.823+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 0x7fdb440a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb50044040 tx=0x7fdb50004770 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.828 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb500041e0 con 0x55c3046443d0 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdb4c01c420 con 0x55c3046443d0 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 10 client.25769 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 10 client.25769 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 20 client.25769 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c304656e00 0x55c3046591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 4 client.25769 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:45.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.824+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3045ae9d0 con 0x55c304656e00 2024-09-05T01:23:45.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.825+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c304656e00 0x55c3046591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.825+0000 7fdb3affd640 10 client.25769 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.826+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c304656e00 0x55c3046591f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660cf9b523897e3e server_cookie=4fb5bf2d0b5ea25e in_seq=0 out_seq=0 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb3affd640 1 -- 172.21.15.129:0/1127608287 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3045ae9d0 con 0x55c304656e00 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb3affd640 10 client.25769 handle_command_reply: tid=0 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 client.25769 shutdown 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 msgr2=0x55c3046529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0126d0 tx=0x7fdb4c012b90 comp rx=0 tx=0).stop 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 shutdown_connections 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c304656e00 0x55c3046591f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046529e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c304643e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1127608287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c304643950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 >> 172.21.15.129:0/1127608287 conn(0x55c304637cf0 msgr2=0x55c30463d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb3affd640 0 client.25769 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 shutdown_connections 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 1 -- 172.21.15.129:0/1127608287 wait complete. 2024-09-05T01:23:45.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.828+0000 7fdb6447e740 20 client.25769 trim_cache size 0 max 16384 2024-09-05T01:23:45.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.861+0000 7fdb6447e740 1 Processor -- start 2024-09-05T01:23:45.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 -- start start 2024-09-05T01:23:45.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c3046527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c30465eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b7d90 con 0x55c304646940 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b5660 con 0x55c3046443d0 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.862+0000 7fdb6447e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c105920 con 0x55c30463b760 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34362/0 (socket says 172.21.15.129:34362) 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 -- 172.21.15.129:0/3691926498 learned_addr learned my addr 172.21.15.129:0/3691926498 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:45.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c3046527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 -- 172.21.15.129:0/3691926498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 msgr2=0x55c3046527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c3046527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 -- 172.21.15.129:0/3691926498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 msgr2=0x55c30465eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c30465eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.863+0000 7fdb5affd640 1 -- 172.21.15.129:0/3691926498 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c304633260 con 0x55c3046443d0 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.864+0000 7fdb5affd640 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c01ca30 tx=0x7fdb4c009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b3685f8a4533dac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.864+0000 7fdb417fa640 1 -- 172.21.15.129:0/3691926498 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f9a0 con 0x55c3046443d0 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.864+0000 7fdb417fa640 1 -- 172.21.15.129:0/3691926498 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb4c00fb40 con 0x55c3046443d0 2024-09-05T01:23:45.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.864+0000 7fdb417fa640 1 -- 172.21.15.129:0/3691926498 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c003a80 con 0x55c3046443d0 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 -- 172.21.15.129:0/3691926498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 msgr2=0x55c3046458c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c01ca30 tx=0x7fdb4c009300 comp rx=0 tx=0).stop 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 -- 172.21.15.129:0/3691926498 shutdown_connections 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c304646940 0x55c30465eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c3046443d0 0x55c3046458c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 --2- 172.21.15.129:0/3691926498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c3046527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 -- 172.21.15.129:0/3691926498 >> 172.21.15.129:0/3691926498 conn(0x55c304637cf0 msgr2=0x55c3046451b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:45.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.865+0000 7fdb6447e740 1 -- 172.21.15.129:0/3691926498 shutdown_connections 2024-09-05T01:23:45.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.866+0000 7fdb6447e740 1 -- 172.21.15.129:0/3691926498 wait complete. 2024-09-05T01:23:45.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.866+0000 7fdb6447e740 1 -- start start 2024-09-05T01:23:45.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.866+0000 7fdb6447e740 1 Processor -- start 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c30464aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c304646940 0x55c3046c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b3b20 con 0x55c3046443d0 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b4f90 con 0x55c304646940 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c1b41f0 con 0x55c30463b760 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb6447e740 10 client.0 _ll_register_callbacks cb 0x7fdb4c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c30464aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38668/0 (socket says 172.21.15.129:38668) 2024-09-05T01:23:45.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb5affd640 1 -- 172.21.15.129:0/1590366526 learned_addr learned my addr 172.21.15.129:0/1590366526 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:45.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.867+0000 7fdb5bfff640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c304646940 0x55c3046c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 -- 172.21.15.129:0/1590366526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 msgr2=0x55c30464aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c30464aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 -- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c304646940 msgr2=0x55c3046c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c304646940 0x55c3046c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 -- 172.21.15.129:0/1590366526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c304638980 con 0x55c3046443d0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb5affd640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0126d0 tx=0x7fdb4c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50d4fdf25bc00f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb3affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00f9a0 con 0x55c3046443d0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.868+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fdb4c018070 con 0x55c3046443d0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.869+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c00fb40 con 0x55c3046443d0 2024-09-05T01:23:45.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.869+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c03d050 con 0x55c3046443d0 2024-09-05T01:23:45.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.870+0000 7fdb6447e740 10 client.16224 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:45.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.870+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3046c1f60 con 0x55c3046443d0 2024-09-05T01:23:45.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb4c00fb40 con 0x55c3046443d0 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fdb4c02a430 con 0x55c3046443d0 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 10 client.16224 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 10 client.16224 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 20 client.16224 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c3046c2640 0x55c3046c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 4 client.16224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55c3045a8d20 con 0x55c3046c2640 2024-09-05T01:23:45.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.871+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c3046c2640 0x55c3046c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:45.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.872+0000 7fdb3affd640 10 client.16224 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:45.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:45.872+0000 7fdb5b7fe640 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c3046c2640 0x55c3046c4a60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=496ea80359f76bc9 server_cookie=345653b3cdca0704 in_seq=0 out_seq=0 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.207+0000 7fdb3affd640 1 -- 172.21.15.129:0/1590366526 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c3045a8d20 con 0x55c3046c2640 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.207+0000 7fdb3affd640 10 client.16224 handle_command_reply: tid=0 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.207+0000 7fdb6447e740 1 client.16224 shutdown 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 msgr2=0x55c30464afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0126d0 tx=0x7fdb4c005270 comp rx=0 tx=0).stop 2024-09-05T01:23:46.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 shutdown_connections 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c3046c2640 0x55c3046c4a60 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c304646940 0x55c3046c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c3046443d0 0x55c30464afd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 --2- 172.21.15.129:0/1590366526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c30463b760 0x55c30464aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 >> 172.21.15.129:0/1590366526 conn(0x55c304637cf0 msgr2=0x55c3046459d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb3affd640 0 client.16224 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 shutdown_connections 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 1 -- 172.21.15.129:0/1590366526 wait complete. 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.208+0000 7fdb6447e740 20 client.16224 trim_cache size 0 max 16384 2024-09-05T01:23:46.210 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 30, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "8a7d67d6": { 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 30, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:23:46.211 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:23:46.212 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 msgr2=0x7fdb440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 0x7fdb440a5050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb50044040 tx=0x7fdb50004770 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 msgr2=0x7fdb5c114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 0x7fdb5c114cb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400dae0 tx=0x7fdb5400b260 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 shutdown_connections 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdb440a2b90 0x7fdb440a5050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdb5c105f40 0x7fdb5c110f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb5c101450 0x7fdb5c1151f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 --2- 172.21.15.129:0/3176991838 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdb5c100b30 0x7fdb5c114cb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:46.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 >> 172.21.15.129:0/3176991838 conn(0x7fdb5c0fc7a0 msgr2=0x7fdb5c1038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:46.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 shutdown_connections 2024-09-05T01:23:46.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:23:46.209+0000 7fdb620a5640 1 -- 172.21.15.129:0/3176991838 wait complete. 2024-09-05T01:23:46.951 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8a7d67d6', quiesce took 6.2 seconds. Will release after: 23.8 2024-09-05T01:23:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:47 smithi129 ceph-mon[30979]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 871 op/s 2024-09-05T01:23:47.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:47 smithi183 ceph-mon[34589]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 871 op/s 2024-09-05T01:23:47.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:47 smithi194 ceph-mon[34941]: pgmap v605: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.1 MiB/s wr, 871 op/s 2024-09-05T01:23:49.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:49 smithi183 ceph-mon[34589]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 892 op/s 2024-09-05T01:23:49.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:49 smithi129 ceph-mon[30979]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 892 op/s 2024-09-05T01:23:49.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:49 smithi194 ceph-mon[34941]: pgmap v606: 129 pgs: 129 active+clean; 1.3 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.6 MiB/s wr, 892 op/s 2024-09-05T01:23:50.212 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 405.0 s, 128.9 tps, lat 108.768 ms stddev 45.749 2024-09-05T01:23:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:51 smithi183 ceph-mon[34589]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 712 op/s 2024-09-05T01:23:51.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:51 smithi129 ceph-mon[30979]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 712 op/s 2024-09-05T01:23:51.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:51 smithi194 ceph-mon[34941]: pgmap v607: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 712 op/s 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 -- 172.21.15.129:0/3469985292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac103710 msgr2=0x7fe1ac109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/3469985292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac103710 0x7fe1ac109fa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009ca0 tx=0x7fe1a402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 -- 172.21.15.129:0/3469985292 shutdown_connections 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/3469985292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac103710 0x7fe1ac109fa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/3469985292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 0x7fe1ac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/3469985292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1ac101b60 0x7fe1ac101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 -- 172.21.15.129:0/3469985292 >> 172.21.15.129:0/3469985292 conn(0x7fe1ac0fd2c0 msgr2=0x7fe1ac0ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.601 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 -- 172.21.15.129:0/3469985292 shutdown_connections 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.598+0000 7fe1b37b0640 1 -- 172.21.15.129:0/3469985292 wait complete. 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 Processor -- start 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 -- start start 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 0x7fe1ac198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1ac103710 0x7fe1ac19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10d670 con 0x7fe1ac101b60 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10d4f0 con 0x7fe1ac102d50 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b37b0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac10d7f0 con 0x7fe1ac103710 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57744/0 (socket says 172.21.15.129:57744) 2024-09-05T01:23:52.602 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 -- 172.21.15.129:0/2758333664 learned_addr learned my addr 172.21.15.129:0/2758333664 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 0x7fe1ac198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 -- 172.21.15.129:0/2758333664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1ac103710 msgr2=0x7fe1ac19f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1ac103710 0x7fe1ac19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 -- 172.21.15.129:0/2758333664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 msgr2=0x7fe1ac198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 0x7fe1ac198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.599+0000 7fe1b154c640 1 -- 172.21.15.129:0/2758333664 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ac19f950 con 0x7fe1ac101b60 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d950 tx=0x7fe1a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1a27fc640 1 -- 172.21.15.129:0/2758333664 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8017020 con 0x7fe1ac101b60 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1a27fc640 1 -- 172.21.15.129:0/2758333664 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a8009c30 con 0x7fe1ac101b60 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ac19fc40 con 0x7fe1ac101b60 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1a27fc640 1 -- 172.21.15.129:0/2758333664 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a800f6b0 con 0x7fe1ac101b60 2024-09-05T01:23:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.600+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ac1a0190 con 0x7fe1ac101b60 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.602+0000 7fe1a27fc640 1 -- 172.21.15.129:0/2758333664 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe1a801a020 con 0x7fe1ac101b60 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.602+0000 7fe1b6625740 1 Processor -- start 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 -- start start 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 0x55dec102ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec102b370 0x55dec102d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dec0f892f0 con 0x55dec10259a0 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dec0f31270 con 0x55dec10262e0 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b6625740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dec0f852b0 con 0x55dec102b370 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57756/0 (socket says 172.21.15.129:57756) 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/2561495074 learned_addr learned my addr 172.21.15.129:0/2561495074 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1a27fc640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 0x7fe1980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec102b370 0x55dec102d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 0x55dec102ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 0x7fe1980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1a27fc640 1 -- 172.21.15.129:0/2758333664 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe1a80cf230 con 0x7fe1ac101b60 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/2561495074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec102b370 msgr2=0x55dec102d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec102b370 0x55dec102d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/2561495074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 msgr2=0x55dec102ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 0x55dec102ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/2561495074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec0f30340 con 0x55dec10259a0 2024-09-05T01:23:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.603+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 0x55dec102ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4004af0 tx=0x7fe1a4032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9091116285d6cdce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1977fe640 1 -- 172.21.15.129:0/2561495074 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4004b40 con 0x55dec10259a0 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1977fe640 1 -- 172.21.15.129:0/2561495074 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a404d070 con 0x55dec10259a0 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1977fe640 1 -- 172.21.15.129:0/2561495074 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40305c0 con 0x55dec10259a0 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1b6625740 1 -- 172.21.15.129:0/2561495074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 msgr2=0x55dec1025da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4004af0 tx=0x7fe1a4032f30 comp rx=0 tx=0).stop 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.604+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 0x7fe1980a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1a404c000 tx=0x7fe1a404d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- 172.21.15.129:0/2561495074 shutdown_connections 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec102b370 0x55dec102d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10262e0 0x55dec102ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2561495074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10259a0 0x55dec1025da0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- 172.21.15.129:0/2561495074 >> 172.21.15.129:0/2561495074 conn(0x55dec1021010 msgr2=0x55dec1021410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- 172.21.15.129:0/2561495074 shutdown_connections 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- 172.21.15.129:0/2561495074 wait complete. 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- start start 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 Processor -- start 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10259a0 0x55dec102fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec102b370 0x55dec1037170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dec102e340 con 0x55dec10262e0 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55dec0f12290 con 0x55dec102b370 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55dec102e4c0 con 0x55dec10259a0 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.605+0000 7fe1b6625740 10 client.0 _ll_register_callbacks cb 0x55dec0f43140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57766/0 (socket says 172.21.15.129:57766) 2024-09-05T01:23:52.613 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 -- 172.21.15.129:0/2961598864 learned_addr learned my addr 172.21.15.129:0/2961598864 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10259a0 0x55dec102fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec102b370 0x55dec1037170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 -- 172.21.15.129:0/2961598864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10259a0 msgr2=0x55dec102fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10259a0 0x55dec102fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 -- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec102b370 msgr2=0x55dec1037170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec102b370 0x55dec1037170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 -- 172.21.15.129:0/2961598864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec0c9c210 con 0x55dec10262e0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00be70 tx=0x7fe19c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ffecc550270114 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c0107e0 con 0x55dec10262e0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe19c010980 con 0x55dec10262e0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a404d070 con 0x55dec10262e0 2024-09-05T01:23:52.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe19c010b80 con 0x55dec10262e0 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b6625740 10 client.16236 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.606+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dec0d68620 con 0x55dec10262e0 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a404d070 con 0x55dec10262e0 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe19c019070 con 0x55dec10262e0 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 10 client.16236 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 20 client.16236 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec103b930 0x55dec103dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 4 client.16236 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dec0f96a20 con 0x55dec103b930 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.607+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec103b930 0x55dec103dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe194ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:52.615 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec103b930 0x55dec103dd60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7160026866fde83f server_cookie=bb596947e8bb142d in_seq=0 out_seq=0 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe194ff9640 1 -- 172.21.15.129:0/2961598864 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dec0f96a20 con 0x55dec103b930 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe194ff9640 10 client.16236 handle_command_reply: tid=0 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 client.16236 shutdown 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 msgr2=0x55dec1030130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00be70 tx=0x7fe19c012820 comp rx=0 tx=0).stop 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 shutdown_connections 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec103b930 0x55dec103dd60 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec102b370 0x55dec1037170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1030130 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe194ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2961598864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10259a0 0x55dec102fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.616 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 >> 172.21.15.129:0/2961598864 conn(0x55dec1021010 msgr2=0x55dec1022820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.608+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 shutdown_connections 2024-09-05T01:23:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.609+0000 7fe1b6625740 1 -- 172.21.15.129:0/2961598864 wait complete. 2024-09-05T01:23:52.617 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.609+0000 7fe1b6625740 20 client.16236 trim_cache size 0 max 16384 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 Processor -- start 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 -- start start 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec1021010 0x55dec1022400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec1037130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1ad120 con 0x55dec10262e0 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1aa9f0 con 0x55dec10259a0 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.623+0000 7fe1b6625740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1a8840 con 0x55dec1021010 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57780/0 (socket says 172.21.15.129:57780) 2024-09-05T01:23:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/1509878176 learned_addr learned my addr 172.21.15.129:0/1509878176 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec1037130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/1509878176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec1021010 msgr2=0x55dec1022400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec1021010 0x55dec1022400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/1509878176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 msgr2=0x55dec1037130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec1037130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 -- 172.21.15.129:0/1509878176 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dec102ebc0 con 0x55dec10262e0 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009dc0 tx=0x7fe1a40099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdf08a38fe7082 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe196ffd640 1 -- 172.21.15.129:0/1509878176 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a4032bc0 con 0x55dec10262e0 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe196ffd640 1 -- 172.21.15.129:0/1509878176 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a4060070 con 0x55dec10262e0 2024-09-05T01:23:52.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.624+0000 7fe196ffd640 1 -- 172.21.15.129:0/1509878176 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40045f0 con 0x55dec10262e0 2024-09-05T01:23:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- 172.21.15.129:0/1509878176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 msgr2=0x55dec1022940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009dc0 tx=0x7fe1a40099c0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- 172.21.15.129:0/1509878176 shutdown_connections 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec10262e0 0x55dec1022940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec1037130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 --2- 172.21.15.129:0/1509878176 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec1021010 0x55dec1022400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- 172.21.15.129:0/1509878176 >> 172.21.15.129:0/1509878176 conn(0x55dec102ae90 msgr2=0x55dec10201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- 172.21.15.129:0/1509878176 shutdown_connections 2024-09-05T01:23:52.631 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- 172.21.15.129:0/1509878176 wait complete. 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 -- start start 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.625+0000 7fe1b6625740 1 Processor -- start 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec10351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10262e0 0x55dec10361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1a8eb0 con 0x55dec1021010 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1aa320 con 0x55dec10259a0 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ac1a9580 con 0x55dec10262e0 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 10 client.0 _ll_register_callbacks cb 0x7fe19c00b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:57796/0 (socket says 172.21.15.129:57796) 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 -- 172.21.15.129:0/2449244704 learned_addr learned my addr 172.21.15.129:0/2449244704 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:23:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec10351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b1d4d640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10262e0 0x55dec10361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 -- 172.21.15.129:0/2449244704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10262e0 msgr2=0x55dec10361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10262e0 0x55dec10361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 -- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 msgr2=0x55dec10351b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec10351b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 -- 172.21.15.129:0/2449244704 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dec10367d0 con 0x55dec1021010 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b154c640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8097a50 tx=0x7fe1a806e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7066146539f34fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe194ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8009de0 con 0x55dec1021010 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe1a8099980 con 0x55dec1021010 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1a4060070 con 0x55dec1021010 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe1a809b680 con 0x55dec1021010 2024-09-05T01:23:52.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 10 client.16242 fetch_fsmap learned FSMap version 30 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.626+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dec10a6a60 con 0x55dec1021010 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1a4060070 con 0x55dec1021010 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe1a8024060 con 0x55dec1021010 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 10 client.16242 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:23:52.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 20 client.16242 populate_metadata read hostname 'smithi129' 2024-09-05T01:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec102b370 0x55dec10a9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 4 client.16242 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dec0f90d70 con 0x55dec102b370 2024-09-05T01:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec102b370 0x55dec10a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:23:52.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe194ff9640 10 client.16242 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:52.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.627+0000 7fe1b0d4b640 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec102b370 0x55dec10a9380 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d58e53af901e0ff server_cookie=c90911554080e045 in_seq=0 out_seq=0 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe194ff9640 1 -- 172.21.15.129:0/2449244704 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dec0f90d70 con 0x55dec102b370 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe194ff9640 10 client.16242 handle_command_reply: tid=0 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 client.16242 shutdown 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 msgr2=0x55dec1034c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8097a50 tx=0x7fe1a806e6b0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 shutdown_connections 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55dec102b370 0x55dec10a9380 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55dec10262e0 0x55dec10361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe194ff9640 0 client.16242 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55dec10259a0 0x55dec10351b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 --2- 172.21.15.129:0/2449244704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dec1021010 0x55dec1034c70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.642+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 >> 172.21.15.129:0/2449244704 conn(0x55dec102ae90 msgr2=0x55dec10201c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 shutdown_connections 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b6625740 1 -- 172.21.15.129:0/2449244704 wait complete. 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b6625740 20 client.16242 trim_cache size 0 max 16384 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:23:52.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:23:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:23:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 msgr2=0x7fe1980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 0x7fe1980a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe1a404c000 tx=0x7fe1a404d000 comp rx=0 tx=0).stop 2024-09-05T01:23:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 msgr2=0x7fe1ac197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:23:52.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe1a800d950 tx=0x7fe1a800de10 comp rx=0 tx=0).stop 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 shutdown_connections 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe1980a2c30 0x7fe1980a50f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe1ac103710 0x7fe1ac19f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe1ac102d50 0x7fe1ac198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 --2- 172.21.15.129:0/2758333664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1ac101b60 0x7fe1ac197d40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 >> 172.21.15.129:0/2758333664 conn(0x7fe1ac0fd2c0 msgr2=0x7fe1ac106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 shutdown_connections 2024-09-05T01:23:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:23:52.643+0000 7fe1b37b0640 1 -- 172.21.15.129:0/2758333664 wait complete. 2024-09-05T01:23:53.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:53 smithi183 ceph-mon[34589]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 605 op/s 2024-09-05T01:23:53.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:53 smithi129 ceph-mon[30979]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 605 op/s 2024-09-05T01:23:53.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:53 smithi194 ceph-mon[34941]: pgmap v608: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.8 MiB/s wr, 605 op/s 2024-09-05T01:23:55.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 410.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:23:55.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:55 smithi183 ceph-mon[34589]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-09-05T01:23:55.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:55 smithi129 ceph-mon[30979]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-09-05T01:23:55.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:55 smithi194 ceph-mon[34941]: pgmap v609: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-09-05T01:23:57.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:57 smithi183 ceph-mon[34589]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-05T01:23:57.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:57 smithi129 ceph-mon[30979]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-05T01:23:57.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:57 smithi194 ceph-mon[34941]: pgmap v610: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-09-05T01:23:59.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:23:59 smithi183 ceph-mon[34589]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T01:23:59.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:23:59 smithi129 ceph-mon[30979]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T01:23:59.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:23:59 smithi194 ceph-mon[34941]: pgmap v611: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T01:24:00.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 415.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:01.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:01 smithi183 ceph-mon[34589]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 713 KiB/s wr, 81 op/s 2024-09-05T01:24:01.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:01 smithi129 ceph-mon[30979]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 713 KiB/s wr, 81 op/s 2024-09-05T01:24:01.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:01 smithi194 ceph-mon[34941]: pgmap v612: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 713 KiB/s wr, 81 op/s 2024-09-05T01:24:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:02 smithi183 ceph-mon[34589]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 346 KiB/s wr, 42 op/s 2024-09-05T01:24:02.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:02 smithi129 ceph-mon[30979]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 346 KiB/s wr, 42 op/s 2024-09-05T01:24:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:02 smithi194 ceph-mon[34941]: pgmap v613: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 346 KiB/s wr, 42 op/s 2024-09-05T01:24:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 420.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:05.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:05 smithi183 ceph-mon[34589]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-09-05T01:24:05.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:05 smithi129 ceph-mon[30979]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-09-05T01:24:05.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:05 smithi194 ceph-mon[34941]: pgmap v614: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 207 KiB/s wr, 36 op/s 2024-09-05T01:24:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.732+0000 7f7a752c3640 1 -- 172.21.15.129:0/1639152750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70101b60 msgr2=0x7f7a70101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.732+0000 7f7a752c3640 1 --2- 172.21.15.129:0/1639152750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70101b60 0x7f7a70101f60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009900 tx=0x7f7a6002a410 comp rx=0 tx=0).stop 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.732+0000 7f7a752c3640 1 -- 172.21.15.129:0/1639152750 shutdown_connections 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.732+0000 7f7a752c3640 1 --2- 172.21.15.129:0/1639152750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70103710 0x7f7a70109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 --2- 172.21.15.129:0/1639152750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70102d50 0x7f7a701031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 --2- 172.21.15.129:0/1639152750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70101b60 0x7f7a70101f60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- 172.21.15.129:0/1639152750 >> 172.21.15.129:0/1639152750 conn(0x7f7a700fd2e0 msgr2=0x7f7a700ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- 172.21.15.129:0/1639152750 shutdown_connections 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- 172.21.15.129:0/1639152750 wait complete. 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 Processor -- start 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- start start 2024-09-05T01:24:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70101b60 0x7f7a70197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 0x7f7a7019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7010d690 con 0x7f7a70102d50 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7010d510 con 0x7f7a70103710 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a752c3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7010d810 con 0x7f7a70101b60 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45206/0 (socket says 172.21.15.129:45206) 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.733+0000 7f7a67fff640 1 -- 172.21.15.129:0/533704642 learned_addr learned my addr 172.21.15.129:0/533704642 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6effd640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70101b60 0x7f7a70197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 0x7f7a7019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 -- 172.21.15.129:0/533704642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70101b60 msgr2=0x7f7a70197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70101b60 0x7f7a70197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 -- 172.21.15.129:0/533704642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 msgr2=0x7f7a7019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 0x7f7a7019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 -- 172.21.15.129:0/533704642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7019f950 con 0x7f7a70102d50 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 0x7f7a7019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a67fff640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d950 tx=0x7f7a5800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6cff9640 1 -- 172.21.15.129:0/533704642 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58017030 con 0x7f7a70102d50 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7019fc40 con 0x7f7a70102d50 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6cff9640 1 -- 172.21.15.129:0/533704642 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a58004020 con 0x7f7a70102d50 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a6cff9640 1 -- 172.21.15.129:0/533704642 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580047a0 con 0x7f7a70102d50 2024-09-05T01:24:05.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.734+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a0160 con 0x7f7a70102d50 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.736+0000 7f7a6cff9640 1 -- 172.21.15.129:0/533704642 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7a580041d0 con 0x7f7a70102d50 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.736+0000 7f7a7805d740 1 Processor -- start 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.736+0000 7f7a7805d740 1 -- start start 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 0x558e79c5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e79bbb2f0 con 0x558e79c579a0 2024-09-05T01:24:05.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e79b63270 con 0x558e79c5d370 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a7805d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558e79bb72b0 con 0x558e79c582e0 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45214/0 (socket says 172.21.15.129:45214) 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/965096920 learned_addr learned my addr 172.21.15.129:0/965096920 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a67fff640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 0x558e79c5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6effd640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/965096920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 msgr2=0x558e79c5cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c5cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/965096920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 msgr2=0x558e79c5f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 0x558e79c5f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/965096920 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e79b62340 con 0x558e79c579a0 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6effd640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:05.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a67fff640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 0x558e79c5f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6cff9640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 0x7f7a5c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.737+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009fd0 tx=0x7f7a68010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67babdd68b5c515 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a6effd640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 0x7f7a5c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a6cff9640 1 -- 172.21.15.129:0/533704642 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f7a580cefc0 con 0x7f7a70102d50 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a657fa640 1 -- 172.21.15.129:0/965096920 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011700 con 0x558e79c579a0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a657fa640 1 -- 172.21.15.129:0/965096920 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a68011d50 con 0x558e79c579a0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a657fa640 1 -- 172.21.15.129:0/965096920 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6800e3c0 con 0x558e79c579a0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a7805d740 1 -- 172.21.15.129:0/965096920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 msgr2=0x558e79c57da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a6effd640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 0x7f7a5c0a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a60004770 tx=0x7f7a60000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.738+0000 7f7a7805d740 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009fd0 tx=0x7f7a68010860 comp rx=0 tx=0).stop 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- 172.21.15.129:0/965096920 shutdown_connections 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c5d370 0x558e79c5f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.747 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- 172.21.15.129:0/965096920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c579a0 0x558e79c57da0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- 172.21.15.129:0/965096920 >> 172.21.15.129:0/965096920 conn(0x558e79c53010 msgr2=0x558e79c53410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- 172.21.15.129:0/965096920 shutdown_connections 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- 172.21.15.129:0/965096920 wait complete. 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- start start 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 Processor -- start 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c61c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c5d370 0x558e79c691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558e79c60340 con 0x558e79c582e0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558e79b44290 con 0x558e79c579a0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558e79c604c0 con 0x558e79c5d370 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a7805d740 10 client.0 _ll_register_callbacks cb 0x558e79b75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45220/0 (socket says 172.21.15.129:45220) 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a67fff640 1 -- 172.21.15.129:0/4121489793 learned_addr learned my addr 172.21.15.129:0/4121489793 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.739+0000 7f7a6effd640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c61c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c5d370 0x558e79c691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 -- 172.21.15.129:0/4121489793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c5d370 msgr2=0x558e79c691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c5d370 0x558e79c691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 -- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 msgr2=0x558e79c61c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c61c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 -- 172.21.15.129:0/4121489793 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e798ce210 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a6effd640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c61c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a67fff640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7a58097d50 tx=0x7f7a58097690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cba84bebff4f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5809ad70 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f7a580ccb70 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a68011d50 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a58099550 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 10 client.16254 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e7999a620 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a68011d50 con 0x558e79c582e0 2024-09-05T01:24:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7a580b4120 con 0x558e79c582e0 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.740+0000 7f7a7805d740 20 client.16254 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79c6d880 0x558e79c6fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a7805d740 4 client.16254 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e79bc8a20 con 0x558e79c6d880 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a6effd640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79c6d880 0x558e79c6fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a6effd640 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79c6d880 0x558e79c6fc70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14855cfa2d5b5b3 server_cookie=3f9574bcdbf13a27 in_seq=0 out_seq=0 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.741+0000 7f7a567fc640 10 client.16254 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a567fc640 1 -- 172.21.15.129:0/4121489793 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e79bc8a20 con 0x558e79c6d880 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a567fc640 10 client.16254 handle_command_reply: tid=0 2024-09-05T01:24:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 client.16254 shutdown 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 msgr2=0x558e79c62160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7a58097d50 tx=0x7f7a58097690 comp rx=0 tx=0).stop 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 shutdown_connections 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79c6d880 0x558e79c6fc70 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c5d370 0x558e79c691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a567fc640 0 client.16254 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c582e0 0x558e79c62160 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 --2- 172.21.15.129:0/4121489793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c61c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 >> 172.21.15.129:0/4121489793 conn(0x558e79c53010 msgr2=0x558e79c54820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 shutdown_connections 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 1 -- 172.21.15.129:0/4121489793 wait complete. 2024-09-05T01:24:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.742+0000 7f7a7805d740 20 client.16254 trim_cache size 0 max 16384 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.756+0000 7f7a7805d740 1 Processor -- start 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.756+0000 7f7a7805d740 1 -- start start 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.758 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701ad0d0 con 0x558e79c5d370 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701aa9a0 con 0x558e79c579a0 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a7805d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701a87f0 con 0x558e79c582e0 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45228/0 (socket says 172.21.15.129:45228) 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/1447551194 learned_addr learned my addr 172.21.15.129:0/1447551194 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6effd640 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a67fff640 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/1447551194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 msgr2=0x558e79c520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/1447551194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 msgr2=0x558e79c57d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c57d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.759 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/1447551194 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e79c70250 con 0x558e79c5d370 2024-09-05T01:24:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009d20 tx=0x7f7a68009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbc8f44c1d3801d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.757+0000 7f7a64ff9640 1 -- 172.21.15.129:0/1447551194 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6802b070 con 0x558e79c5d370 2024-09-05T01:24:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a64ff9640 1 -- 172.21.15.129:0/1447551194 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6801bda0 con 0x558e79c5d370 2024-09-05T01:24:05.760 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a64ff9640 1 -- 172.21.15.129:0/1447551194 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68011860 con 0x558e79c5d370 2024-09-05T01:24:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- 172.21.15.129:0/1447551194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 msgr2=0x558e79c54400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009d20 tx=0x7f7a68009d50 comp rx=0 tx=0).stop 2024-09-05T01:24:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- 172.21.15.129:0/1447551194 shutdown_connections 2024-09-05T01:24:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79c54400 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c582e0 0x558e79c520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 --2- 172.21.15.129:0/1447551194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c579a0 0x558e79c57d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- 172.21.15.129:0/1447551194 >> 172.21.15.129:0/1447551194 conn(0x558e79c53010 msgr2=0x558e79c519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- 172.21.15.129:0/1447551194 shutdown_connections 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- 172.21.15.129:0/1447551194 wait complete. 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.758+0000 7f7a7805d740 1 -- start start 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 Processor -- start 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 0x558e79cd39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c582e0 0x558e79cd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701a8e60 con 0x558e79c5d370 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701aa2d0 con 0x558e79c582e0 2024-09-05T01:24:05.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701a9530 con 0x558e79c579a0 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 10 client.0 _ll_register_callbacks cb 0x7f7a58005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45242/0 (socket says 172.21.15.129:45242) 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/900644817 learned_addr learned my addr 172.21.15.129:0/900644817 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6effd640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 0x558e79cd39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a67fff640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c582e0 0x558e79cd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/900644817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 msgr2=0x558e79cd39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 0x558e79cd39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c582e0 msgr2=0x558e79cd3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c582e0 0x558e79cd3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 -- 172.21.15.129:0/900644817 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e79cdb570 con 0x558e79c5d370 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6effd640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 0x558e79cd39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a6f7fe640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800a010 tx=0x7f7a68004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617b288e335bcdb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a567fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6802b070 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f7a680115e0 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.759+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6801bda0 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7a6801b8c0 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 10 client.16260 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e79cdc880 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6801bda0 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f7a68025070 con 0x558e79c5d370 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 10 client.16260 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 20 client.16260 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79cdcfe0 0x558e79cdf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 4 client.16260 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:24:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e79bc2d70 con 0x558e79cdcfe0 2024-09-05T01:24:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.760+0000 7f7a6effd640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79cdcfe0 0x558e79cdf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.761+0000 7f7a567fc640 10 client.16260 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.761+0000 7f7a6effd640 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79cdcfe0 0x558e79cdf3d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59ec539f6d1c2d71 server_cookie=929732db3ebd67dc in_seq=0 out_seq=0 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.772+0000 7f7a567fc640 1 -- 172.21.15.129:0/900644817 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e79bc2d70 con 0x558e79cdcfe0 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.772+0000 7f7a567fc640 10 client.16260 handle_command_reply: tid=0 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.772+0000 7f7a7805d740 1 client.16260 shutdown 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 msgr2=0x558e79cdaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800a010 tx=0x7f7a68004e10 comp rx=0 tx=0).stop 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 shutdown_connections 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558e79cdcfe0 0x558e79cdf3d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558e79c5d370 0x558e79cdaf60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558e79c582e0 0x558e79cd3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 --2- 172.21.15.129:0/900644817 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558e79c579a0 0x558e79cd39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a567fc640 0 client.16260 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 >> 172.21.15.129:0/900644817 conn(0x558e79c53010 msgr2=0x558e79c521c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 shutdown_connections 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 1 -- 172.21.15.129:0/900644817 wait complete. 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a7805d740 20 client.16260 trim_cache size 0 max 16384 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:24:05.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:24:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 msgr2=0x7f7a5c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 0x7f7a5c0a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7a60004770 tx=0x7f7a60000bc0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.773+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 msgr2=0x7f7a70198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800d950 tx=0x7f7a5800de10 comp rx=0 tx=0).stop 2024-09-05T01:24:05.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 shutdown_connections 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7a5c0a2c30 0x7f7a5c0a50f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a70103710 0x7f7a7019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7a70102d50 0x7f7a70198280 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 --2- 172.21.15.129:0/533704642 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7a70101b60 0x7f7a70197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 >> 172.21.15.129:0/533704642 conn(0x7f7a700fd2e0 msgr2=0x7f7a701060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 shutdown_connections 2024-09-05T01:24:05.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:05.774+0000 7f7a752c3640 1 -- 172.21.15.129:0/533704642 wait complete. 2024-09-05T01:24:07.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:07 smithi183 ceph-mon[34589]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 32 op/s 2024-09-05T01:24:07.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:07 smithi129 ceph-mon[30979]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 32 op/s 2024-09-05T01:24:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:07 smithi194 ceph-mon[34941]: pgmap v615: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 195 KiB/s wr, 32 op/s 2024-09-05T01:24:09.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:09 smithi183 ceph-mon[34589]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-09-05T01:24:09.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:09 smithi129 ceph-mon[30979]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-09-05T01:24:09.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:09 smithi194 ceph-mon[34941]: pgmap v616: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 226 KiB/s wr, 39 op/s 2024-09-05T01:24:10.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 425.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:10.752 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:24:11.017 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:11.140 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:11 smithi129 ceph-mon[30979]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 210 KiB/s wr, 33 op/s 2024-09-05T01:24:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:11 smithi183 ceph-mon[34589]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 210 KiB/s wr, 33 op/s 2024-09-05T01:24:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:11 smithi194 ceph-mon[34941]: pgmap v617: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 210 KiB/s wr, 33 op/s 2024-09-05T01:24:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 -- 172.21.15.129:0/311220874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041082a0 msgr2=0x7f970410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 --2- 172.21.15.129:0/311220874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041082a0 0x7f970410ec40 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009930 tx=0x7f96f402a420 comp rx=0 tx=0).stop 2024-09-05T01:24:11.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 -- 172.21.15.129:0/311220874 shutdown_connections 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 --2- 172.21.15.129:0/311220874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97041082a0 0x7f970410ec40 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 --2- 172.21.15.129:0/311220874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041036e0 0x7f9704107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 --2- 172.21.15.129:0/311220874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9704102d30 0x7f9704103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 -- 172.21.15.129:0/311220874 >> 172.21.15.129:0/311220874 conn(0x7f97040fc7d0 msgr2=0x7f97040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 -- 172.21.15.129:0/311220874 shutdown_connections 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.497+0000 7f9708b87640 1 -- 172.21.15.129:0/311220874 wait complete. 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 Processor -- start 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 -- start start 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 0x7f97041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041082a0 0x7f970419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:11.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9704111f50 con 0x7f9704102d30 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9704111dd0 con 0x7f97041082a0 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9708b87640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97041120d0 con 0x7f97041036e0 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f970259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f970259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45252/0 (socket says 172.21.15.129:45252) 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f970259c640 1 -- 172.21.15.129:0/2660641854 learned_addr learned my addr 172.21.15.129:0/2660641854 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.498+0000 7f9701d9b640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 0x7f97041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f9702d9d640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041082a0 0x7f970419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 -- 172.21.15.129:0/2660641854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 msgr2=0x7f97041989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 0x7f97041989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 -- 172.21.15.129:0/2660641854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041082a0 msgr2=0x7f970419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041082a0 0x7f970419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 -- 172.21.15.129:0/2660641854 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970419d420 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f9701d9b640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 0x7f97041989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f970259c640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00ea80 tx=0x7f96fc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00ce40 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96fc005020 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96fc0116d0 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970419d710 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.499+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97041a4ff0 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.500+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9704103110 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.501+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f96fc0051d0 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.502+0000 7f96fb7fe640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 0x7f96f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.502+0000 7f9701d9b640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 0x7f96f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.502+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f96fc0cfbe0 con 0x7f9704102d30 2024-09-05T01:24:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.502+0000 7f9701d9b640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 0x7f96f00a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005fd0 tx=0x7f96ec005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:11.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.505+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96fc016030 con 0x7f9704102d30 2024-09-05T01:24:11.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.704+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97041997d0 con 0x7f9704102d30 2024-09-05T01:24:11.706 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.705+0000 7f96fb7fe640 1 -- 172.21.15.129:0/2660641854 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f97041997d0 con 0x7f9704102d30 2024-09-05T01:24:11.707 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:24:11.707 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:24:11.707 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 msgr2=0x7f96f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 0x7f96f00a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005fd0 tx=0x7f96ec005850 comp rx=0 tx=0).stop 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 msgr2=0x7f9704198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00ea80 tx=0x7f96fc00ef40 comp rx=0 tx=0).stop 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 shutdown_connections 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f96f00a2b80 0x7f96f00a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97041082a0 0x7f970419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.709 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97041036e0 0x7f97041989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 --2- 172.21.15.129:0/2660641854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9704102d30 0x7f9704198470 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 >> 172.21.15.129:0/2660641854 conn(0x7f97040fc7d0 msgr2=0x7f9704105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 shutdown_connections 2024-09-05T01:24:11.710 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:11.707+0000 7f9708b87640 1 -- 172.21.15.129:0/2660641854 wait complete. 2024-09-05T01:24:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:12 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2660641854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:12 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2660641854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:12 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2660641854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:13.153 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 8a7d67d6 --release --await' 2024-09-05T01:24:13.153 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 8a7d67d6 --release --await 2024-09-05T01:24:13.409 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:13 smithi183 ceph-mon[34589]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 195 KiB/s wr, 33 op/s 2024-09-05T01:24:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:13 smithi129 ceph-mon[30979]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 195 KiB/s wr, 33 op/s 2024-09-05T01:24:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:13 smithi194 ceph-mon[34941]: pgmap v618: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 195 KiB/s wr, 33 op/s 2024-09-05T01:24:15.182 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:15 smithi129 ceph-mon[30979]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-09-05T01:24:15.183 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:24:15.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 430.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:15.290 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 -- 172.21.15.129:0/807530745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181082a0 msgr2=0x7fbf1810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/807530745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181082a0 0x7fbf1810ec40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009cd0 tx=0x7fbf1002a8e0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 -- 172.21.15.129:0/807530745 shutdown_connections 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/807530745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181082a0 0x7fbf1810ec40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/807530745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf181036e0 0x7fbf18107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/807530745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf18102d30 0x7fbf18103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 -- 172.21.15.129:0/807530745 >> 172.21.15.129:0/807530745 conn(0x7fbf180fc7d0 msgr2=0x7fbf180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 -- 172.21.15.129:0/807530745 shutdown_connections 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.286+0000 7fbf1df5a640 1 -- 172.21.15.129:0/807530745 wait complete. 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 Processor -- start 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 -- start start 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 0x7fbf18100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.291 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf181082a0 0x7fbf18101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18111f50 con 0x7fbf181036e0 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18111dd0 con 0x7fbf18102d30 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.287+0000 7fbf1df5a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181120d0 con 0x7fbf181082a0 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 0x7fbf18100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41818/0 (socket says 172.21.15.129:41818) 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf16ffd640 1 -- 172.21.15.129:0/2023869032 learned_addr learned my addr 172.21.15.129:0/2023869032 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf17fff640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf181082a0 0x7fbf18101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 -- 172.21.15.129:0/2023869032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf181082a0 msgr2=0x7fbf18101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf181082a0 0x7fbf18101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.292 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 -- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 msgr2=0x7fbf18100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 -- 172.21.15.129:0/2023869032 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf181a6910 con 0x7fbf18102d30 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf177fe640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 0x7fbf18100990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf08005fd0 tx=0x7fbf0800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf14ff9640 1 -- 172.21.15.129:0/2023869032 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08009c30 con 0x7fbf18102d30 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf14ff9640 1 -- 172.21.15.129:0/2023869032 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf08009dd0 con 0x7fbf18102d30 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf181a6c00 con 0x7fbf18102d30 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf14ff9640 1 -- 172.21.15.129:0/2023869032 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08003e40 con 0x7fbf18102d30 2024-09-05T01:24:15.293 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.288+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf181a7140 con 0x7fbf18102d30 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.291+0000 7fbf14ff9640 1 -- 172.21.15.129:0/2023869032 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbf08004000 con 0x7fbf18102d30 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf14ff9640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 0x7fbf040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 Processor -- start 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 0x7fbf040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 -- start start 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf14ff9640 1 -- 172.21.15.129:0/2023869032 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbf080cf000 con 0x7fbf18102d30 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.295 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d9924b620 con 0x559d992e6980 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d991f2e50 con 0x559d992e6060 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf20333740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d992475e0 con 0x559d992f1240 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:43584/0 (socket says 172.21.15.129:43584) 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.292+0000 7fbf17fff640 1 -- 172.21.15.129:0/2204911934 learned_addr learned my addr 172.21.15.129:0/2204911934 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 0x7fbf040a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbf18102420 tx=0x7fbf00005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 -- 172.21.15.129:0/2204911934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 msgr2=0x559d992e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 -- 172.21.15.129:0/2204911934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 msgr2=0x559d992f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992f0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 -- 172.21.15.129:0/2204911934 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d991e3d40 con 0x559d992f1240 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf17fff640 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009cd0 tx=0x7fbf10009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b3385183a35b71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf0dffb640 1 -- 172.21.15.129:0/2204911934 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10049070 con 0x559d992f1240 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf0dffb640 1 -- 172.21.15.129:0/2204911934 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf10036040 con 0x559d992f1240 2024-09-05T01:24:15.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.293+0000 7fbf0dffb640 1 -- 172.21.15.129:0/2204911934 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf100042a0 con 0x559d992f1240 2024-09-05T01:24:15.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- 172.21.15.129:0/2204911934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 msgr2=0x559d992f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009cd0 tx=0x7fbf10009a20 comp rx=0 tx=0).stop 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- 172.21.15.129:0/2204911934 shutdown_connections 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f3630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 --2- 172.21.15.129:0/2204911934 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- 172.21.15.129:0/2204911934 >> 172.21.15.129:0/2204911934 conn(0x559d992e25f0 msgr2=0x559d992e29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- 172.21.15.129:0/2204911934 shutdown_connections 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- 172.21.15.129:0/2204911934 wait complete. 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.294+0000 7fbf20333740 1 -- start start 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 Processor -- start 2024-09-05T01:24:15.301 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d992f3eb0 con 0x559d992e6980 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559d991d3f10 con 0x559d992e6060 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559d992f4030 con 0x559d992f1240 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf20333740 10 client.0 _ll_register_callbacks cb 0x559d991f7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41848/0 (socket says 172.21.15.129:41848) 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf177fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47848/0 (socket says 172.21.15.129:47848) 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 -- 172.21.15.129:0/3845140757 learned_addr learned my addr 172.21.15.129:0/3845140757 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf17fff640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 -- 172.21.15.129:0/3845140757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 msgr2=0x559d992eeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992eeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 -- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 msgr2=0x559d992ee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf16ffd640 1 -- 172.21.15.129:0/3845140757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d98f5a580 con 0x559d992e6980 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.295+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000fdb0 tx=0x7fbf00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d559caca1ebd67b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0002b060 con 0x559d992e6980 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbf0001db20 con 0x559d992e6980 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf10036040 con 0x559d992e6980 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf00032050 con 0x559d992e6980 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf20333740 10 client.16278 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:15.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d99024960 con 0x559d992e6980 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf10036040 con 0x559d992e6980 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbf0001c400 con 0x559d992e6980 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf20333740 10 client.16278 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.296+0000 7fbf20333740 10 client.16278 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf20333740 20 client.16278 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d99300ec0 0x559d99303310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf20333740 4 client.16278 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d99258b90 con 0x559d99300ec0 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d99300ec0 0x559d99303310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbef6ffd640 10 client.16278 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.297+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d99300ec0 0x559d99303310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb56e01e1bd03844 server_cookie=484b34d361c818b in_seq=0 out_seq=0 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3845140757 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d99258b90 con 0x559d99300ec0 2024-09-05T01:24:15.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbef6ffd640 10 client.16278 handle_command_reply: tid=0 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 client.16278 shutdown 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 msgr2=0x559d992ee610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000fdb0 tx=0x7fbf00004730 comp rx=0 tx=0).stop 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 shutdown_connections 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d99300ec0 0x559d99303310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992eeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6980 0x559d992ee610 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 --2- 172.21.15.129:0/3845140757 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6060 0x559d992ee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 >> 172.21.15.129:0/3845140757 conn(0x559d992e25f0 msgr2=0x559d992e8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbef6ffd640 0 client.16278 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 shutdown_connections 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.298+0000 7fbf20333740 1 -- 172.21.15.129:0/3845140757 wait complete. 2024-09-05T01:24:15.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.299+0000 7fbf20333740 20 client.16278 trim_cache size 0 max 16384 2024-09-05T01:24:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.315+0000 7fbf20333740 1 Processor -- start 2024-09-05T01:24:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.315+0000 7fbf20333740 1 -- start start 2024-09-05T01:24:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992e6060 0x559d99308be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d99309120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 0x559d992eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181b3880 con 0x559d992f1240 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181b1150 con 0x559d992e6980 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf20333740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181aefa0 con 0x559d992e6060 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 0x559d992eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d99309120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 0x559d992eb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41850/0 (socket says 172.21.15.129:41850) 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf17fff640 1 -- 172.21.15.129:0/1193609375 learned_addr learned my addr 172.21.15.129:0/1193609375 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf177fe640 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992e6060 0x559d99308be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 -- 172.21.15.129:0/1193609375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992e6060 msgr2=0x559d99308be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992e6060 0x559d99308be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 -- 172.21.15.129:0/1193609375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 msgr2=0x559d992eb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 0x559d992eb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 -- 172.21.15.129:0/1193609375 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d992e67b0 con 0x559d992e6980 2024-09-05T01:24:15.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d99309120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf0001c850 tx=0x7fbf00005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=386bc8f3f6fdafb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.316+0000 7fbf0d7fa640 1 -- 172.21.15.129:0/1193609375 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0001c400 con 0x559d992e6980 2024-09-05T01:24:15.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf0d7fa640 1 -- 172.21.15.129:0/1193609375 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf0001c5a0 con 0x559d992e6980 2024-09-05T01:24:15.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf0d7fa640 1 -- 172.21.15.129:0/1193609375 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00026680 con 0x559d992e6980 2024-09-05T01:24:15.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 -- 172.21.15.129:0/1193609375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 msgr2=0x559d99309120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d99309120 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf0001c850 tx=0x7fbf00005850 comp rx=0 tx=0).stop 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 -- 172.21.15.129:0/1193609375 shutdown_connections 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992f1240 0x559d992eb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d99309120 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 --2- 172.21.15.129:0/1193609375 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992e6060 0x559d99308be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 -- 172.21.15.129:0/1193609375 >> 172.21.15.129:0/1193609375 conn(0x559d992e25f0 msgr2=0x559d992ec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.317+0000 7fbf20333740 1 -- 172.21.15.129:0/1193609375 shutdown_connections 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 -- 172.21.15.129:0/1193609375 wait complete. 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 -- start start 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 Processor -- start 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 0x559d992f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.323 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d992f9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181af610 con 0x559d992e6060 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181b0a80 con 0x559d992e6980 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbf181afce0 con 0x559d992f1240 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.318+0000 7fbf20333740 10 client.0 _ll_register_callbacks cb 0x7fbf0000f960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:43610/0 (socket says 172.21.15.129:43610) 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 -- 172.21.15.129:0/3422789644 learned_addr learned my addr 172.21.15.129:0/3422789644 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf16ffd640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d992f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 -- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 msgr2=0x559d992f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d992f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.324 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 -- 172.21.15.129:0/3422789644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 msgr2=0x559d992f92c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 0x559d992f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 0x559d992f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 -- 172.21.15.129:0/3422789644 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d992f47b0 con 0x559d992f1240 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 0x559d992f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf17fff640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf10033040 tx=0x7fbf10003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46e0df518f48915c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10049040 con 0x559d992f1240 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fbf10031530 con 0x559d992f1240 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.319+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0001c5a0 con 0x559d992f1240 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbf1005f050 con 0x559d992f1240 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 10 client.25768 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:15.325 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d9936c390 con 0x559d992f1240 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf0001c5a0 con 0x559d992f1240 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fbf10030070 con 0x559d992f1240 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 10 client.25768 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 10 client.25768 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 20 client.25768 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d9936c670 0x559d9936ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 4 client.25768 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8a7d67d6", "release": true, "await": true}] 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8a7d67d6", "release": true, "await": true}) -- 0x559d992530a0 con 0x559d9936c670 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.320+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d9936c670 0x559d9936ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.321+0000 7fbef6ffd640 10 client.25768 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:15.326 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.321+0000 7fbf177fe640 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d9936c670 0x559d9936ea60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e7c35c3bf7420e server_cookie=7cc490fa2703e023 in_seq=0 out_seq=0 2024-09-05T01:24:15.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.373+0000 7fbef6ffd640 1 -- 172.21.15.129:0/3422789644 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559d992530a0 con 0x559d9936c670 2024-09-05T01:24:15.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.373+0000 7fbef6ffd640 10 client.25768 handle_command_reply: tid=0 2024-09-05T01:24:15.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.373+0000 7fbf20333740 1 client.25768 shutdown 2024-09-05T01:24:15.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.373+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 msgr2=0x559d992f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.373+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf10033040 tx=0x7fbf10003970 comp rx=0 tx=0).stop 2024-09-05T01:24:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 shutdown_connections 2024-09-05T01:24:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559d9936c670 0x559d9936ea60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbef6ffd640 0 client.25768 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559d992f1240 0x559d992f4270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559d992e6980 0x559d992f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 --2- 172.21.15.129:0/3422789644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d992e6060 0x559d992f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.375+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 >> 172.21.15.129:0/3422789644 conn(0x559d992e25f0 msgr2=0x559d992e7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.376+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 shutdown_connections 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.376+0000 7fbf20333740 1 -- 172.21.15.129:0/3422789644 wait complete. 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:24:15.378 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 32, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "8a7d67d6": { 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 32, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:24:15.379 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:15.380 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.376+0000 7fbf20333740 20 client.25768 trim_cache size 0 max 16384 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 msgr2=0x7fbf040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 0x7fbf040a5040 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbf18102420 tx=0x7fbf00005f90 comp rx=0 tx=0).stop 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 msgr2=0x7fbf18100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 0x7fbf18100990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbf08005fd0 tx=0x7fbf0800dde0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 shutdown_connections 2024-09-05T01:24:15.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbf040a2b80 0x7fbf040a5040 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbf181082a0 0x7fbf18101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf181036e0 0x7fbf18100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 --2- 172.21.15.129:0/2023869032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbf18102d30 0x7fbf18100990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.377+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 >> 172.21.15.129:0/2023869032 conn(0x7fbf180fc7d0 msgr2=0x7fbf18105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.378+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 shutdown_connections 2024-09-05T01:24:15.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:15.378+0000 7fbf1df5a640 1 -- 172.21.15.129:0/2023869032 wait complete. 2024-09-05T01:24:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:15 smithi183 ceph-mon[34589]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-09-05T01:24:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:24:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:15 smithi194 ceph-mon[34941]: pgmap v619: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 223 KiB/s wr, 36 op/s 2024-09-05T01:24:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:24:16.086 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:24:16.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:24:16.087 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:16 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:24:16.141 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8a7d67d6', total seconds elapsed: 35.4 2024-09-05T01:24:16.141 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T01:24:16.142 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:24:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:24:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:24:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:16 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:24:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:24:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:24:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:16 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:24:16.619 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 -- 172.21.15.129:0/1603523281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44102790 msgr2=0x7feb441067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 --2- 172.21.15.129:0/1603523281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44102790 0x7feb441067e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7feb34009860 tx=0x7feb3402a340 comp rx=0 tx=0).stop 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 -- 172.21.15.129:0/1603523281 shutdown_connections 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 --2- 172.21.15.129:0/1603523281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44106f10 0x7feb4410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 --2- 172.21.15.129:0/1603523281 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44102790 0x7feb441067e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 --2- 172.21.15.129:0/1603523281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44101de0 0x7feb441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 -- 172.21.15.129:0/1603523281 >> 172.21.15.129:0/1603523281 conn(0x7feb440fd8c0 msgr2=0x7feb440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 -- 172.21.15.129:0/1603523281 shutdown_connections 2024-09-05T01:24:17.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.215+0000 7feb4b273640 1 -- 172.21.15.129:0/1603523281 wait complete. 2024-09-05T01:24:17.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.216+0000 7feb4b273640 1 Processor -- start 2024-09-05T01:24:17.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 -- start start 2024-09-05T01:24:17.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44101de0 0x7feb44075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:17.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4880e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4880e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47894/0 (socket says 172.21.15.129:47894) 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44106f10 0x7feb4406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4880e640 1 -- 172.21.15.129:0/1308279675 learned_addr learned my addr 172.21.15.129:0/1308279675 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb4406e4a0 con 0x7feb44101de0 2024-09-05T01:24:17.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb4406e320 con 0x7feb44102790 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.217+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7feb4406e620 con 0x7feb44106f10 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb49810640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44106f10 0x7feb4406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4900f640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44101de0 0x7feb44075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 -- 172.21.15.129:0/1308279675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44106f10 msgr2=0x7feb4406fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44106f10 0x7feb4406fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 -- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44101de0 msgr2=0x7feb44075a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44101de0 0x7feb44075a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb44070480 con 0x7feb44102790 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.218+0000 7feb4880e640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb3402a8b0 tx=0x7feb34030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.219+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34004460 con 0x7feb44102790 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.219+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb34030900 con 0x7feb44102790 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.219+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb340052f0 con 0x7feb44102790 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.219+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb44070710 con 0x7feb44102790 2024-09-05T01:24:17.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.219+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4419a4f0 con 0x7feb44102790 2024-09-05T01:24:17.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.221+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb441032c0 con 0x7feb44102790 2024-09-05T01:24:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.222+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7feb34030450 con 0x7feb44102790 2024-09-05T01:24:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.223+0000 7feb3a7fc640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 0x7feb300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.224+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7feb340f78f0 con 0x7feb44102790 2024-09-05T01:24:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.224+0000 7feb4900f640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 0x7feb300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:17.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.224+0000 7feb4900f640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 0x7feb300a5260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb40009bd0 tx=0x7feb40005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:17.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.228+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb340c43e0 con 0x7feb44102790 2024-09-05T01:24:17.400 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:17 smithi129 ceph-mon[30979]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 172 KiB/s wr, 29 op/s 2024-09-05T01:24:17.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:17 smithi183 ceph-mon[34589]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 172 KiB/s wr, 29 op/s 2024-09-05T01:24:17.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:17 smithi194 ceph-mon[34941]: pgmap v620: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 172 KiB/s wr, 29 op/s 2024-09-05T01:24:17.633 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:24:17.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.628+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb4410a150 con 0x7feb44102790 2024-09-05T01:24:17.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.630+0000 7feb3a7fc640 1 -- 172.21.15.129:0/1308279675 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7feb340c3ce0 con 0x7feb44102790 2024-09-05T01:24:17.634 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:24:17.634 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:24:17.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.635+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 msgr2=0x7feb300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:17.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.635+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 0x7feb300a5260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7feb40009bd0 tx=0x7feb40005f90 comp rx=0 tx=0).stop 2024-09-05T01:24:17.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.635+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 msgr2=0x7feb44075fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:17.637 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.635+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb3402a8b0 tx=0x7feb34030d80 comp rx=0 tx=0).stop 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 shutdown_connections 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7feb300a2da0 0x7feb300a5260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7feb44106f10 0x7feb4406fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb44102790 0x7feb44075fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 --2- 172.21.15.129:0/1308279675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb44101de0 0x7feb44075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:17.638 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.636+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 >> 172.21.15.129:0/1308279675 conn(0x7feb440fd8c0 msgr2=0x7feb440fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:17.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.637+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 shutdown_connections 2024-09-05T01:24:17.639 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:17.637+0000 7feb4b273640 1 -- 172.21.15.129:0/1308279675 wait complete. 2024-09-05T01:24:18.345 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:18 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1308279675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:18.403 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T01:24:18.403 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T01:24:18.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:18 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1308279675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:18 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1308279675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:18.864 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.938+0000 7f19e579d640 1 -- 172.21.15.129:0/481851080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0072cb0 msgr2=0x7f19e00730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.938+0000 7f19e579d640 1 --2- 172.21.15.129:0/481851080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0072cb0 0x7f19e00730b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009860 tx=0x7f19d002a340 comp rx=0 tx=0).stop 2024-09-05T01:24:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.938+0000 7f19e579d640 1 -- 172.21.15.129:0/481851080 shutdown_connections 2024-09-05T01:24:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.938+0000 7f19e579d640 1 --2- 172.21.15.129:0/481851080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e0114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.938+0000 7f19e579d640 1 --2- 172.21.15.129:0/481851080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0073680 0x7f19e010b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.939+0000 7f19e579d640 1 --2- 172.21.15.129:0/481851080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0072cb0 0x7f19e00730b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.939+0000 7f19e579d640 1 -- 172.21.15.129:0/481851080 >> 172.21.15.129:0/481851080 conn(0x7f19e006eb50 msgr2=0x7f19e0070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.940+0000 7f19e579d640 1 -- 172.21.15.129:0/481851080 shutdown_connections 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.940+0000 7f19e579d640 1 -- 172.21.15.129:0/481851080 wait complete. 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.940+0000 7f19e579d640 1 Processor -- start 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 -- start start 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0073680 0x7f19e01a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e007a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0117ee0 con 0x7f19e0073680 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0117d60 con 0x7f19e0072cb0 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19e579d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0118060 con 0x7f19e010bba0 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47910/0 (socket says 172.21.15.129:47910) 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.941+0000 7f19dffff640 1 -- 172.21.15.129:0/3716339732 learned_addr learned my addr 172.21.15.129:0/3716339732 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e007a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19dffff640 1 -- 172.21.15.129:0/3716339732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 msgr2=0x7f19e007a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19dffff640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e007a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19dffff640 1 -- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0073680 msgr2=0x7f19e01a2b30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19dffff640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0073680 0x7f19e01a2b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19dffff640 1 -- 172.21.15.129:0/3716339732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e007aaf0 con 0x7f19e0072cb0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.942+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e007a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.943+0000 7f19dffff640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035040 tx=0x7f19d0030d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.943+0000 7f19ddffb640 1 -- 172.21.15.129:0/3716339732 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0040020 con 0x7f19e0072cb0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.943+0000 7f19ddffb640 1 -- 172.21.15.129:0/3716339732 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d0038040 con 0x7f19e0072cb0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.943+0000 7f19ddffb640 1 -- 172.21.15.129:0/3716339732 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0037070 con 0x7f19e0072cb0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.943+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e007ad80 con 0x7f19e0072cb0 2024-09-05T01:24:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.944+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e007b160 con 0x7f19e0072cb0 2024-09-05T01:24:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.948+0000 7f19ddffb640 1 -- 172.21.15.129:0/3716339732 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f19d0043020 con 0x7f19e0072cb0 2024-09-05T01:24:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.949+0000 7f19e986e740 1 Processor -- start 2024-09-05T01:24:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.949+0000 7f19e986e740 1 -- start start 2024-09-05T01:24:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dbb70 0x555d6c1dbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1e1540 0x555d6c1e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555d6c13f2f0 con 0x555d6c1e1540 2024-09-05T01:24:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d6c0e7270 con 0x555d6c1dc4b0 2024-09-05T01:24:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e986e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d6c13b2b0 con 0x555d6c1dbb70 2024-09-05T01:24:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47912/0 (socket says 172.21.15.129:47912) 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19e4f9c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dbb70 0x555d6c1dbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19dffff640 1 -- 172.21.15.129:0/2031388735 learned_addr learned my addr 172.21.15.129:0/2031388735 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.950+0000 7f19d77fe640 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1e1540 0x555d6c1e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 -- 172.21.15.129:0/2031388735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dbb70 msgr2=0x555d6c1dbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dbb70 0x555d6c1dbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 -- 172.21.15.129:0/2031388735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1e1540 msgr2=0x555d6c1e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1e1540 0x555d6c1e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 -- 172.21.15.129:0/2031388735 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d6c0e6340 con 0x555d6c1dc4b0 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19dffff640 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19d00f7360 tx=0x7f19d00f7450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a61cc494bd01d77 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19ddffb640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 0x7f19cc0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 0x7f19cc0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19d67fc640 1 -- 172.21.15.129:0/2031388735 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00b7a70 con 0x555d6c1dc4b0 2024-09-05T01:24:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.951+0000 7f19ddffb640 1 -- 172.21.15.129:0/3716339732 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f19d00ef3e0 con 0x7f19e0072cb0 2024-09-05T01:24:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.952+0000 7f19d67fc640 1 -- 172.21.15.129:0/2031388735 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d00fc030 con 0x555d6c1dc4b0 2024-09-05T01:24:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.952+0000 7f19d67fc640 1 -- 172.21.15.129:0/2031388735 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00c2800 con 0x555d6c1dc4b0 2024-09-05T01:24:18.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.952+0000 7f19e986e740 1 -- 172.21.15.129:0/2031388735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 msgr2=0x555d6c1e0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.952+0000 7f19e986e740 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19d00f7360 tx=0x7f19d00f7450 comp rx=0 tx=0).stop 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.952+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 0x7f19cc0a5250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009fd0 tx=0x7f19d8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 -- 172.21.15.129:0/2031388735 shutdown_connections 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1e1540 0x555d6c1e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e0f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 --2- 172.21.15.129:0/2031388735 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dbb70 0x555d6c1dbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 -- 172.21.15.129:0/2031388735 >> 172.21.15.129:0/2031388735 conn(0x555d6c1d71e0 msgr2=0x555d6c1d75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 -- 172.21.15.129:0/2031388735 shutdown_connections 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 -- 172.21.15.129:0/2031388735 wait complete. 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.953+0000 7f19e986e740 1 -- start start 2024-09-05T01:24:18.956 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.954+0000 7f19e986e740 1 Processor -- start 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555d6c0e7270 con 0x555d6c1dc4b0 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e4f9c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47920/0 (socket says 172.21.15.129:47920) 2024-09-05T01:24:18.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19dffff640 1 -- 172.21.15.129:0/74220179 learned_addr learned my addr 172.21.15.129:0/74220179 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555d6c0c8290 con 0x555d6c1dbb70 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.955+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555d6c1e42d0 con 0x555d6c1e1540 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e986e740 10 client.0 _ll_register_callbacks cb 0x555d6c0f9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e4f9c640 1 -- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 msgr2=0x555d6c1e5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e4f9c640 1 -- 172.21.15.129:0/74220179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 msgr2=0x555d6c1e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.956+0000 7f19e4f9c640 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d6be52210 con 0x555d6c1e1540 2024-09-05T01:24:18.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.957+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1ece80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005fd0 tx=0x7f19d8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bf0ba4ca2ec541b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:18.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.957+0000 7f19bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.957+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d80044d0 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.957+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f19d8018030 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.957+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d00fc030 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.958+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19d8004670 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.958+0000 7f19e986e740 10 client.25786 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.958+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555d6bf1e620 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d00fc030 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f19d8011440 con 0x555d6c1e1540 2024-09-05T01:24:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 10 client.25786 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 10 client.25786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 20 client.25786 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c1f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 4 client.25786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.959+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d6c14ca20 con 0x555d6c1f15a0 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.960+0000 7f19dffff640 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c1f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.961+0000 7f19dffff640 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c1f39f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476ffd1474e1f54d server_cookie=772a58b31da2ff38 in_seq=0 out_seq=0 2024-09-05T01:24:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.961+0000 7f19bf7fe640 10 client.25786 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.962+0000 7f19bf7fe640 1 -- 172.21.15.129:0/74220179 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d6c14ca20 con 0x555d6c1f15a0 2024-09-05T01:24:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.962+0000 7f19bf7fe640 10 client.25786 handle_command_reply: tid=0 2024-09-05T01:24:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.962+0000 7f19e986e740 1 client.25786 shutdown 2024-09-05T01:24:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.963+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 msgr2=0x555d6c1ece80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:18.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.963+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1ece80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f19d8005fd0 tx=0x7f19d8004ab0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 shutdown_connections 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c1f39f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19bf7fe640 0 client.25786 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1ece80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 --2- 172.21.15.129:0/74220179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 >> 172.21.15.129:0/74220179 conn(0x555d6c1d71e0 msgr2=0x555d6c1d89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 shutdown_connections 2024-09-05T01:24:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 1 -- 172.21.15.129:0/74220179 wait complete. 2024-09-05T01:24:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.966+0000 7f19e986e740 20 client.25786 trim_cache size 0 max 16384 2024-09-05T01:24:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.998+0000 7f19e986e740 1 Processor -- start 2024-09-05T01:24:19.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.998+0000 7f19e986e740 1 -- start start 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.999+0000 7f19e986e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.999+0000 7f19e986e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.999+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.999+0000 7f19dffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47928/0 (socket says 172.21.15.129:47928) 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:18.999+0000 7f19dffff640 1 -- 172.21.15.129:0/3148683779 learned_addr learned my addr 172.21.15.129:0/3148683779 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.000+0000 7f19e986e740 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.000+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19e01c8b60 con 0x555d6c1dc4b0 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.000+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.000+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19e01c6430 con 0x555d6c1dbb70 2024-09-05T01:24:19.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.000+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0083af0 con 0x555d6c1e1540 2024-09-05T01:24:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 -- 172.21.15.129:0/3148683779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 msgr2=0x555d6c1e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 -- 172.21.15.129:0/3148683779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 msgr2=0x555d6c1ece40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 -- 172.21.15.129:0/3148683779 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d6c1d7e70 con 0x555d6c1dbb70 2024-09-05T01:24:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19dffff640 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19d00f7950 tx=0x7f19d0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12e35463ca0eac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19d5ffb640 1 -- 172.21.15.129:0/3148683779 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00c2a30 con 0x555d6c1dbb70 2024-09-05T01:24:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19d5ffb640 1 -- 172.21.15.129:0/3148683779 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19d00fc040 con 0x555d6c1dbb70 2024-09-05T01:24:19.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.001+0000 7f19d5ffb640 1 -- 172.21.15.129:0/3148683779 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d00c2ca0 con 0x555d6c1dbb70 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.002+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 msgr2=0x555d6c1e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.002+0000 7f19e986e740 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19d00f7950 tx=0x7f19d0009490 comp rx=0 tx=0).stop 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 shutdown_connections 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1e1540 0x555d6c1e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ece40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 --2- 172.21.15.129:0/3148683779 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1dbb70 0x555d6c1e5900 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 >> 172.21.15.129:0/3148683779 conn(0x555d6c1d71e0 msgr2=0x555d6c1dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 shutdown_connections 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 -- 172.21.15.129:0/3148683779 wait complete. 2024-09-05T01:24:19.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 -- start start 2024-09-05T01:24:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.003+0000 7f19e986e740 1 Processor -- start 2024-09-05T01:24:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dbb70 0x555d6c1ea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19d77fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:43666/0 (socket says 172.21.15.129:43666) 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e4f9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e4f9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47944/0 (socket says 172.21.15.129:47944) 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19d77fe640 1 -- 172.21.15.129:0/2216917265 learned_addr learned my addr 172.21.15.129:0/2216917265 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0084160 con 0x555d6c1dbb70 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f19e01c5d60 con 0x555d6c1e1540 2024-09-05T01:24:19.009 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f19e0084830 con 0x555d6c1dc4b0 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.004+0000 7f19dffff640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dbb70 0x555d6c1ea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 -- 172.21.15.129:0/2216917265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 msgr2=0x555d6c1ea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 -- 172.21.15.129:0/2216917265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dbb70 msgr2=0x555d6c1ea130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dbb70 0x555d6c1ea130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d6c1ebc90 con 0x555d6c1e1540 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e986e740 10 client.0 _ll_register_callbacks cb 0x7f19d80126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19d00fc040 con 0x555d6c1e1540 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19d77fe640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19e4f9c640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19d8011890 tx=0x7f19d80039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=235e7a1e91cd8fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19bf7fe640 10 client.25853 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:24:19.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19bf7fe640 1 -- 172.21.15.129:0/2216917265 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d8040050 con 0x555d6c1e1540 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19bf7fe640 1 -- 172.21.15.129:0/2216917265 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f19d8011c90 con 0x555d6c1e1540 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.005+0000 7f19bf7fe640 1 -- 172.21.15.129:0/2216917265 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f19d80044a0 con 0x555d6c1e1540 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 10 client.25853 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=30}) -- 0x555d6c1ecef0 con 0x555d6c1e1540 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19bf7fe640 1 -- 172.21.15.129:0/2216917265 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f19d8032470 con 0x555d6c1e1540 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 10 client.25853 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 20 client.25853 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c25e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 4 client.25853 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.006+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555d6c146d70 con 0x555d6c1f15a0 2024-09-05T01:24:19.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.007+0000 7f19dffff640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c25e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.007+0000 7f19dffff640 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c25e7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c055ee7491dfe0f4 server_cookie=4a4865d5850b0302 in_seq=0 out_seq=0 2024-09-05T01:24:19.012 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.007+0000 7f19bf7fe640 10 client.25853 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.055+0000 7f19bf7fe640 1 -- 172.21.15.129:0/2216917265 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555d6c146d70 con 0x555d6c1f15a0 2024-09-05T01:24:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.055+0000 7f19bf7fe640 10 client.25853 handle_command_reply: tid=0 2024-09-05T01:24:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 client.25853 shutdown 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 msgr2=0x555d6c1eb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f19d8011890 tx=0x7f19d80039e0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 shutdown_connections 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555d6c1f15a0 0x555d6c25e7a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555d6c1e1540 0x555d6c1eb680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555d6c1dc4b0 0x555d6c1ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.056+0000 7f19bf7fe640 0 client.25853 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.057+0000 7f19e986e740 1 --2- 172.21.15.129:0/2216917265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555d6c1dbb70 0x555d6c1ea130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.057+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 >> 172.21.15.129:0/2216917265 conn(0x555d6c1d71e0 msgr2=0x555d6c1dd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.057+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 shutdown_connections 2024-09-05T01:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.057+0000 7f19e986e740 1 -- 172.21.15.129:0/2216917265 wait complete. 2024-09-05T01:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.057+0000 7f19e986e740 20 client.25853 trim_cache size 0 max 16384 2024-09-05T01:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:24:19.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 msgr2=0x7f19cc0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 0x7f19cc0a5250 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009fd0 tx=0x7f19d8009500 comp rx=0 tx=0).stop 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 msgr2=0x7f19e01a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19d0035040 tx=0x7f19d0030d80 comp rx=0 tx=0).stop 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 shutdown_connections 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f19cc0a2d90 0x7f19cc0a5250 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f19e010bba0 0x7f19e007a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19e0073680 0x7f19e01a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 --2- 172.21.15.129:0/3716339732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19e0072cb0 0x7f19e01a25f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 >> 172.21.15.129:0/3716339732 conn(0x7f19e006eb50 msgr2=0x7f19e0070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 shutdown_connections 2024-09-05T01:24:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:19.060+0000 7f19e579d640 1 -- 172.21.15.129:0/3716339732 wait complete. 2024-09-05T01:24:19.308 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:19 smithi129 ceph-mon[30979]: pgmap v621: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 589 KiB/s wr, 82 op/s 2024-09-05T01:24:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:19 smithi183 ceph-mon[34589]: pgmap v621: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 589 KiB/s wr, 82 op/s 2024-09-05T01:24:19.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.462+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/3460523209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 msgr2=0x7fb19c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.462+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/3460523209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c101200 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb184009890 tx=0x7fb18402a410 comp rx=0 tx=0).stop 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.464+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/3460523209 shutdown_connections 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.464+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/3460523209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb19c102ef0 0x7fb19c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.464+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/3460523209 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.464+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/3460523209 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c101200 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.464+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/3460523209 >> 172.21.15.129:0/3460523209 conn(0x7fb19c0fc7d0 msgr2=0x7fb19c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.466+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/3460523209 shutdown_connections 2024-09-05T01:24:19.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.466+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/3460523209 wait complete. 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 Processor -- start 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 -- start start 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb19c102ef0 0x7fb19c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c118540 con 0x7fb19c102ef0 2024-09-05T01:24:19.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1183c0 con 0x7fb19c102570 2024-09-05T01:24:19.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a2e8a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1186c0 con 0x7fb19c100e20 2024-09-05T01:24:19.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb1a0c26640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c06bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47952/0 (socket says 172.21.15.129:47952) 2024-09-05T01:24:19.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.467+0000 7fb19bfff640 1 -- 172.21.15.129:0/2218392143 learned_addr learned my addr 172.21.15.129:0/2218392143 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 -- 172.21.15.129:0/2218392143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 msgr2=0x7fb19c06bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c06bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 -- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb19c102ef0 msgr2=0x7fb19c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb19c102ef0 0x7fb19c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 -- 172.21.15.129:0/2218392143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19c06ca40 con 0x7fb19c100e20 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.468+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c073d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb184009890 tx=0x7fb184030ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.469+0000 7fb199ffb640 1 -- 172.21.15.129:0/2218392143 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184040070 con 0x7fb19c100e20 2024-09-05T01:24:19.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.469+0000 7fb199ffb640 1 -- 172.21.15.129:0/2218392143 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb184004460 con 0x7fb19c100e20 2024-09-05T01:24:19.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.469+0000 7fb199ffb640 1 -- 172.21.15.129:0/2218392143 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840046c0 con 0x7fb19c100e20 2024-09-05T01:24:19.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.469+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19c1b3810 con 0x7fb19c100e20 2024-09-05T01:24:19.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.470+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19c1b3ca0 con 0x7fb19c100e20 2024-09-05T01:24:19.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.473+0000 7fb1a5263740 1 Processor -- start 2024-09-05T01:24:19.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.474+0000 7fb1a5263740 1 -- start start 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 0x560ca63ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63d7940 0x560ca63d9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560ca63322a0 con 0x560ca63cc760 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ca62d9b10 con 0x560ca63cd080 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a5263740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ca632e260 con 0x560ca63d7940 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a0c26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a0c26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47964/0 (socket says 172.21.15.129:47964) 2024-09-05T01:24:19.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a0c26640 1 -- 172.21.15.129:0/1759221294 learned_addr learned my addr 172.21.15.129:0/1759221294 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.475+0000 7fb1a1427640 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 0x560ca63ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 -- 172.21.15.129:0/1759221294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63d7940 msgr2=0x560ca63d9d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:24:19.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63d7940 0x560ca63d9d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.477 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 -- 172.21.15.129:0/1759221294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 msgr2=0x560ca63ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 0x560ca63ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 -- 172.21.15.129:0/1759221294 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca60a3b90 con 0x560ca63cd080 2024-09-05T01:24:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a1427640 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 0x560ca63ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1840f3f80 tx=0x7fb1840f81a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13faea99ac94b304 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.476+0000 7fb199ffb640 1 -- 172.21.15.129:0/2218392143 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb184043070 con 0x7fb19c100e20 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.477+0000 7fb192ffd640 1 -- 172.21.15.129:0/1759221294 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840f8de0 con 0x560ca63cd080 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.477+0000 7fb192ffd640 1 -- 172.21.15.129:0/1759221294 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1840fd070 con 0x560ca63cd080 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.477+0000 7fb192ffd640 1 -- 172.21.15.129:0/1759221294 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1841006a0 con 0x560ca63cd080 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb199ffb640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 0x7fb1880a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 -- 172.21.15.129:0/1759221294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 msgr2=0x560ca63d7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb1840f3f80 tx=0x7fb1840f81a0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb199ffb640 1 -- 172.21.15.129:0/2218392143 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb1840efea0 con 0x7fb19c100e20 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a1427640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 0x7fb1880a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 -- 172.21.15.129:0/1759221294 shutdown_connections 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63d7940 0x560ca63d9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d7400 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1759221294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63cc760 0x560ca63ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.478+0000 7fb1a5263740 1 -- 172.21.15.129:0/1759221294 >> 172.21.15.129:0/1759221294 conn(0x560ca63c8cf0 msgr2=0x560ca63c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.480 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.479+0000 7fb1a5263740 1 -- 172.21.15.129:0/1759221294 shutdown_connections 2024-09-05T01:24:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.480+0000 7fb1a1427640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 0x7fb1880a5210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb1940099a0 tx=0x7fb194011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.481+0000 7fb1a5263740 1 -- 172.21.15.129:0/1759221294 wait complete. 2024-09-05T01:24:19.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.481+0000 7fb1a5263740 1 -- start start 2024-09-05T01:24:19.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.481+0000 7fb1a5263740 1 Processor -- start 2024-09-05T01:24:19.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63d4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560ca63da2d0 con 0x560ca63d7940 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ca62ba6e0 con 0x560ca63cd080 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x560ca63da450 con 0x560ca63cc760 2024-09-05T01:24:19.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb1a5263740 10 client.0 _ll_register_callbacks cb 0x560ca62dea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47968/0 (socket says 172.21.15.129:47968) 2024-09-05T01:24:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.482+0000 7fb19bfff640 1 -- 172.21.15.129:0/234462379 learned_addr learned my addr 172.21.15.129:0/234462379 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 -- 172.21.15.129:0/234462379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 msgr2=0x560ca63d4820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:24:19.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63d4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 -- 172.21.15.129:0/234462379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 msgr2=0x560ca63d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 -- 172.21.15.129:0/234462379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca63e3730 con 0x560ca63cd080 2024-09-05T01:24:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb19bfff640 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0116f0 tx=0x7fb18c011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110120d87e26325 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.483+0000 7fb17bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00f920 con 0x560ca63cd080 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1840fd070 con 0x560ca63cd080 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb18c00fac0 con 0x560ca63cd080 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb18c00fcc0 con 0x560ca63cd080 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb1a5263740 10 client.25865 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:19.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ca63e6120 con 0x560ca63cd080 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1840fd070 con 0x560ca63cd080 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.484+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb18c018070 con 0x560ca63cd080 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 10 client.25865 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 10 client.25865 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 20 client.25865 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca63e7ab0 0x560ca63e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 4 client.25865 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca633f9d0 con 0x560ca63e7ab0 2024-09-05T01:24:19.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.485+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca63e7ab0 0x560ca63e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.486+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca63e7ab0 0x560ca63e9f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87f302d8a6e93cd server_cookie=1d63943ef3a57702 in_seq=0 out_seq=0 2024-09-05T01:24:19.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.486+0000 7fb17bfff640 10 client.25865 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb17bfff640 1 -- 172.21.15.129:0/234462379 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ca633f9d0 con 0x560ca63e7ab0 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb17bfff640 10 client.25865 handle_command_reply: tid=0 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 client.25865 shutdown 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 msgr2=0x560ca63d4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0116f0 tx=0x7fb18c011bb0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 shutdown_connections 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca63e7ab0 0x560ca63e9f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63d4d60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.487+0000 7fb1a5263740 1 --2- 172.21.15.129:0/234462379 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63d4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.488+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 >> 172.21.15.129:0/234462379 conn(0x560ca63c8cf0 msgr2=0x560ca63cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.488+0000 7fb17bfff640 0 client.25865 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.488+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 shutdown_connections 2024-09-05T01:24:19.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.488+0000 7fb1a5263740 1 -- 172.21.15.129:0/234462379 wait complete. 2024-09-05T01:24:19.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.489+0000 7fb1a5263740 20 client.25865 trim_cache size 0 max 16384 2024-09-05T01:24:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:19 smithi194 ceph-mon[34941]: pgmap v621: 129 pgs: 129 active+clean; 1.3 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 589 KiB/s wr, 82 op/s 2024-09-05T01:24:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 Processor -- start 2024-09-05T01:24:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 -- start start 2024-09-05T01:24:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63efd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.522 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1c06c0 con 0x560ca63d7940 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1bdf90 con 0x560ca63cd080 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.520+0000 7fb1a5263740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1bbde0 con 0x560ca63cc760 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47978/0 (socket says 172.21.15.129:47978) 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 -- 172.21.15.129:0/1966447238 learned_addr learned my addr 172.21.15.129:0/1966447238 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb1a1427640 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 -- 172.21.15.129:0/1966447238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 msgr2=0x560ca63c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 -- 172.21.15.129:0/1966447238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 msgr2=0x560ca63efd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.523 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63efd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 -- 172.21.15.129:0/1966447238 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca63c8b20 con 0x560ca63cd080 2024-09-05T01:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.521+0000 7fb19bfff640 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009230 tx=0x7fb18c009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6613f39c376ef00b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.522+0000 7fb1927fc640 1 -- 172.21.15.129:0/1966447238 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c02c030 con 0x560ca63cd080 2024-09-05T01:24:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.522+0000 7fb1927fc640 1 -- 172.21.15.129:0/1966447238 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb18c004320 con 0x560ca63cd080 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.522+0000 7fb1927fc640 1 -- 172.21.15.129:0/1966447238 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c000e60 con 0x560ca63cd080 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 -- 172.21.15.129:0/1966447238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 msgr2=0x560ca63ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009230 tx=0x7fb18c009c80 comp rx=0 tx=0).stop 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 -- 172.21.15.129:0/1966447238 shutdown_connections 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63efd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cd080 0x560ca63ef840 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009230 tx=0x7fb18c009c80 comp rx=0 tx=0).stop 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 --2- 172.21.15.129:0/1966447238 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63cc760 0x560ca63c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.523+0000 7fb1a5263740 1 -- 172.21.15.129:0/1966447238 >> 172.21.15.129:0/1966447238 conn(0x560ca63c8cf0 msgr2=0x560ca63c7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.525+0000 7fb1a5263740 1 -- 172.21.15.129:0/1966447238 shutdown_connections 2024-09-05T01:24:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.525+0000 7fb1a5263740 1 -- 172.21.15.129:0/1966447238 wait complete. 2024-09-05T01:24:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.526+0000 7fb1a5263740 1 -- start start 2024-09-05T01:24:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.526+0000 7fb1a5263740 1 Processor -- start 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63df170 0x560ca63df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1bc450 con 0x560ca63d7940 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1bd8c0 con 0x560ca63cc760 2024-09-05T01:24:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a0c26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a5263740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb19c1bcb20 con 0x560ca63df170 2024-09-05T01:24:19.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a0c26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:47986/0 (socket says 172.21.15.129:47986) 2024-09-05T01:24:19.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a0c26640 1 -- 172.21.15.129:0/383063970 learned_addr learned my addr 172.21.15.129:0/383063970 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:19.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.527+0000 7fb1a1427640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63df170 0x560ca63df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a5263740 10 client.0 _ll_register_callbacks cb 0x7fb18c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:19.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 -- 172.21.15.129:0/383063970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63df170 msgr2=0x560ca63df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63df170 0x560ca63df5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 -- 172.21.15.129:0/383063970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 msgr2=0x560ca63dc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 -- 172.21.15.129:0/383063970 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca63dfbe0 con 0x560ca63cc760 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb1a0c26640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b8c60 tx=0x7fb18402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e37a72cf6d28f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb17bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.528+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1840b8530 con 0x560ca63cc760 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.529+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb1840ba330 con 0x560ca63cc760 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.529+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb18c004320 con 0x560ca63cc760 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.529+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb1840bbbf0 con 0x560ca63cc760 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb1a5263740 10 client.25871 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:19.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ca64537c0 con 0x560ca63cc760 2024-09-05T01:24:19.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1840fd040 con 0x560ca63cc760 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb1840b8900 con 0x560ca63cc760 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb1a5263740 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb1a5263740 10 client.25871 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.530+0000 7fb1a5263740 20 client.25871 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.531+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca6453e10 0x560ca6456220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.531+0000 7fb1a5263740 4 client.25871 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.531+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fb18c004320 con 0x560ca6453e10 2024-09-05T01:24:19.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.531+0000 7fb19bfff640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca6453e10 0x560ca6456220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:19.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.533+0000 7fb19bfff640 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca6453e10 0x560ca6456220 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00c3649235357d server_cookie=b09caf0bbae03e2c in_seq=0 out_seq=0 2024-09-05T01:24:19.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.534+0000 7fb17bfff640 10 client.25871 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.831 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb17bfff640 1 -- 172.21.15.129:0/383063970 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7fb18c004320 con 0x560ca6453e10 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb17bfff640 10 client.25871 handle_command_reply: tid=0 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 client.25871 shutdown 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 msgr2=0x560ca63dbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb1840b8c60 tx=0x7fb18402a410 comp rx=0 tx=0).stop 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 shutdown_connections 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x560ca6453e10 0x560ca6456220 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x560ca63df170 0x560ca63df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560ca63d7940 0x560ca63dc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 --2- 172.21.15.129:0/383063970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ca63cc760 0x560ca63dbad0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb17bfff640 0 client.25871 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 >> 172.21.15.129:0/383063970 conn(0x560ca63c8cf0 msgr2=0x560ca63d5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 shutdown_connections 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 1 -- 172.21.15.129:0/383063970 wait complete. 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.830+0000 7fb1a5263740 20 client.25871 trim_cache size 0 max 16384 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 34, 2024-09-05T01:24:19.833 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "ba18b76": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 34, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:19.834 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:24:19.835 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 msgr2=0x7fb1880a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 0x7fb1880a5210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb1940099a0 tx=0x7fb194011040 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 msgr2=0x7fb19c073d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c073d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb184009890 tx=0x7fb184030ce0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 shutdown_connections 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1880a2d50 0x7fb1880a5210 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb19c102ef0 0x7fb19c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb19c102570 0x7fb19c06bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 --2- 172.21.15.129:0/2218392143 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb19c100e20 0x7fb19c073d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 >> 172.21.15.129:0/2218392143 conn(0x7fb19c0fc7d0 msgr2=0x7fb19c0fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 shutdown_connections 2024-09-05T01:24:19.836 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:19.831+0000 7fb1a2e8a640 1 -- 172.21.15.129:0/2218392143 wait complete. 2024-09-05T01:24:20.155 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:20 smithi129 ceph-mon[30979]: pgmap v622: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-05T01:24:20.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 435.0 s, 751.9 tps, lat 896.940 ms stddev 4740.436 2024-09-05T01:24:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:20 smithi183 ceph-mon[34589]: pgmap v622: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-05T01:24:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:20 smithi194 ceph-mon[34941]: pgmap v622: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-09-05T01:24:20.503 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ba18b76', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-05T01:24:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:23 smithi183 ceph-mon[34589]: pgmap v623: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-05T01:24:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:23 smithi129 ceph-mon[30979]: pgmap v623: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-05T01:24:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:23 smithi194 ceph-mon[34941]: pgmap v623: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.1 MiB/s wr, 233 op/s 2024-09-05T01:24:25.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:25 smithi183 ceph-mon[34589]: pgmap v624: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-09-05T01:24:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:25 smithi129 ceph-mon[30979]: pgmap v624: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-09-05T01:24:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:25 smithi194 ceph-mon[34941]: pgmap v624: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-09-05T01:24:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:27 smithi183 ceph-mon[34589]: pgmap v625: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-09-05T01:24:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:27 smithi129 ceph-mon[30979]: pgmap v625: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-09-05T01:24:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:27 smithi194 ceph-mon[34941]: pgmap v625: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.4 MiB/s wr, 277 op/s 2024-09-05T01:24:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:29 smithi183 ceph-mon[34589]: pgmap v626: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-09-05T01:24:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:29 smithi129 ceph-mon[30979]: pgmap v626: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-09-05T01:24:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:29 smithi194 ceph-mon[34941]: pgmap v626: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-09-05T01:24:30.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:31.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:31 smithi183 ceph-mon[34589]: pgmap v627: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-05T01:24:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:31 smithi129 ceph-mon[30979]: pgmap v627: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-05T01:24:31.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:31 smithi194 ceph-mon[34941]: pgmap v627: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 -- 172.21.15.129:0/2083344565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c101c30 msgr2=0x7fe45c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 --2- 172.21.15.129:0/2083344565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c101c30 0x7fe45c1020b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009900 tx=0x7fe44c02a410 comp rx=0 tx=0).stop 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 -- 172.21.15.129:0/2083344565 shutdown_connections 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 --2- 172.21.15.129:0/2083344565 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c1025f0 0x7fe45c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 --2- 172.21.15.129:0/2083344565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c101c30 0x7fe45c1020b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 --2- 172.21.15.129:0/2083344565 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe45c100a40 0x7fe45c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.172 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 -- 172.21.15.129:0/2083344565 >> 172.21.15.129:0/2083344565 conn(0x7fe45c0fc1e0 msgr2=0x7fe45c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 -- 172.21.15.129:0/2083344565 shutdown_connections 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.169+0000 7fe46394b640 1 -- 172.21.15.129:0/2083344565 wait complete. 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 Processor -- start 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 -- start start 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 0x7fe45c109a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe45c1025f0 0x7fe45c109f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10d6c0 con 0x7fe45c100a40 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10d540 con 0x7fe45c101c30 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe46394b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c10d840 con 0x7fe45c1025f0 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58592/0 (socket says 172.21.15.129:58592) 2024-09-05T01:24:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 -- 172.21.15.129:0/2177475600 learned_addr learned my addr 172.21.15.129:0/2177475600 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe460ee6640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 0x7fe45c109a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 -- 172.21.15.129:0/2177475600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe45c1025f0 msgr2=0x7fe45c109f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe45c1025f0 0x7fe45c109f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 -- 172.21.15.129:0/2177475600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 msgr2=0x7fe45c109a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 0x7fe45c109a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe4616e7640 1 -- 172.21.15.129:0/2177475600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c10a7d0 con 0x7fe45c100a40 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.170+0000 7fe460ee6640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 0x7fe45c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d950 tx=0x7fe45800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe4527fc640 1 -- 172.21.15.129:0/2177475600 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458017070 con 0x7fe45c100a40 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe4527fc640 1 -- 172.21.15.129:0/2177475600 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe458009c30 con 0x7fe45c100a40 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c1a1e30 con 0x7fe45c100a40 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe4527fc640 1 -- 172.21.15.129:0/2177475600 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45800f620 con 0x7fe45c100a40 2024-09-05T01:24:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.171+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c1a21e0 con 0x7fe45c100a40 2024-09-05T01:24:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe4527fc640 1 -- 172.21.15.129:0/2177475600 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe45801a070 con 0x7fe45c100a40 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 Processor -- start 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 -- start start 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc89bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc89e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55abcc7fa2f0 con 0x55abcc8969a0 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abcc7a2270 con 0x55abcc89c370 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe466322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abcc7f62b0 con 0x55abcc8972e0 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58600/0 (socket says 172.21.15.129:58600) 2024-09-05T01:24:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe461ee8640 1 -- 172.21.15.129:0/697494033 learned_addr learned my addr 172.21.15.129:0/697494033 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe4616e7640 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc89bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.174+0000 7fe460ee6640 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc89e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe4527fc640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 0x7fe4480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 -- 172.21.15.129:0/697494033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 msgr2=0x55abcc89bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc89bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 -- 172.21.15.129:0/697494033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 msgr2=0x55abcc89e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc89e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 -- 172.21.15.129:0/697494033 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcc7a1340 con 0x55abcc8969a0 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe4527fc640 1 -- 172.21.15.129:0/2177475600 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe4580cef50 con 0x7fe45c100a40 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 0x7fe4480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe461ee8640 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe454009d80 tx=0x7fe45400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192e2469e41cc55 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe4477fe640 1 -- 172.21.15.129:0/697494033 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45401e950 con 0x55abcc8969a0 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe4477fe640 1 -- 172.21.15.129:0/697494033 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45401eb70 con 0x55abcc8969a0 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.175+0000 7fe4477fe640 1 -- 172.21.15.129:0/697494033 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454010610 con 0x55abcc8969a0 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- 172.21.15.129:0/697494033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 msgr2=0x55abcc896da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe461ee8640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 0x7fe4480a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe45401d000 tx=0x7fe454010d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe454009d80 tx=0x7fe45400e6c0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- 172.21.15.129:0/697494033 shutdown_connections 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc89e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc89bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 --2- 172.21.15.129:0/697494033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8969a0 0x55abcc896da0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.182 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- 172.21.15.129:0/697494033 >> 172.21.15.129:0/697494033 conn(0x55abcc892010 msgr2=0x55abcc892410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- 172.21.15.129:0/697494033 shutdown_connections 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- 172.21.15.129:0/697494033 wait complete. 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 -- start start 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.176+0000 7fe466322740 1 Processor -- start 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc8a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8972e0 0x55abcc8a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55abcc89f340 con 0x55abcc89c370 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55abcc783290 con 0x55abcc8972e0 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55abcc89f4c0 con 0x55abcc8969a0 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe466322740 10 client.0 _ll_register_callbacks cb 0x55abcc7b4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:32.183 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58610/0 (socket says 172.21.15.129:58610) 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 -- 172.21.15.129:0/2196591226 learned_addr learned my addr 172.21.15.129:0/2196591226 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe460ee6640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8972e0 0x55abcc8a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc8a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 -- 172.21.15.129:0/2196591226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 msgr2=0x55abcc8a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc8a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 -- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8972e0 msgr2=0x55abcc8a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8972e0 0x55abcc8a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.177+0000 7fe461ee8640 1 -- 172.21.15.129:0/2196591226 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcc50d210 con 0x55abcc89c370 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe461ee8640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe45400e700 tx=0x7fe454014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118edfab2fc17c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:32.184 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45402f070 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe4540105d0 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe45401eb70 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe45403b050 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 10 client.16317 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abcc5d9620 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe45401eb70 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe45400f040 con 0x55abcc89c370 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 10 client.16317 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 20 client.16317 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc8ae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 4 client.16317 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.178+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abcc807a20 con 0x55abcc8ac570 2024-09-05T01:24:32.185 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.179+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc8ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.179+0000 7fe4616e7640 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc8ae960 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d34e45da22622840 server_cookie=f3f63fe907e680bb in_seq=0 out_seq=0 2024-09-05T01:24:32.186 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.179+0000 7fe444ff9640 10 client.16317 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe444ff9640 1 -- 172.21.15.129:0/2196591226 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abcc807a20 con 0x55abcc8ac570 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe444ff9640 10 client.16317 handle_command_reply: tid=0 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 client.16317 shutdown 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 msgr2=0x55abcc8a80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe45400e700 tx=0x7fe454014000 comp rx=0 tx=0).stop 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 shutdown_connections 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc8ae960 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a80e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe444ff9640 0 client.16317 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:32.187 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8972e0 0x55abcc8a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 --2- 172.21.15.129:0/2196591226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc8a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 >> 172.21.15.129:0/2196591226 conn(0x55abcc892010 msgr2=0x55abcc893820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 shutdown_connections 2024-09-05T01:24:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 1 -- 172.21.15.129:0/2196591226 wait complete. 2024-09-05T01:24:32.188 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.180+0000 7fe466322740 20 client.16317 trim_cache size 0 max 16384 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.194+0000 7fe466322740 1 Processor -- start 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.194+0000 7fe466322740 1 -- start start 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc893400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc893940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1af390 con 0x55abcc8972e0 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1acc60 con 0x55abcc89c370 2024-09-05T01:24:32.200 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe466322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1aaab0 con 0x55abcc8969a0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58620/0 (socket says 172.21.15.129:58620) 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 -- 172.21.15.129:0/3213623016 learned_addr learned my addr 172.21.15.129:0/3213623016 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 -- 172.21.15.129:0/3213623016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 msgr2=0x55abcc893400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc893400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 -- 172.21.15.129:0/3213623016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 msgr2=0x55abcc893940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc893940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 -- 172.21.15.129:0/3213623016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcc8aecd0 con 0x55abcc8972e0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe460ee6640 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005940 tx=0x7fe44c032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526801c38af8e823 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe446ffd640 1 -- 172.21.15.129:0/3213623016 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c035040 con 0x55abcc8972e0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe446ffd640 1 -- 172.21.15.129:0/3213623016 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe44c031840 con 0x55abcc8972e0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.195+0000 7fe446ffd640 1 -- 172.21.15.129:0/3213623016 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c004920 con 0x55abcc8972e0 2024-09-05T01:24:32.201 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- 172.21.15.129:0/3213623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 msgr2=0x55abcc8a80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe44c005940 tx=0x7fe44c032dd0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- 172.21.15.129:0/3213623016 shutdown_connections 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc89c370 0x55abcc893940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc8972e0 0x55abcc8a80a0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 --2- 172.21.15.129:0/3213623016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8969a0 0x55abcc893400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- 172.21.15.129:0/3213623016 >> 172.21.15.129:0/3213623016 conn(0x55abcc892010 msgr2=0x55abcc8911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- 172.21.15.129:0/3213623016 shutdown_connections 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- 172.21.15.129:0/3213623016 wait complete. 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 -- start start 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.196+0000 7fe466322740 1 Processor -- start 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 0x55abcc8a5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc8a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.202 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1ab120 con 0x55abcc89c370 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1ac590 con 0x55abcc8969a0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c1ab7f0 con 0x55abcc8972e0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe466322740 10 client.0 _ll_register_callbacks cb 0x7fe4580d2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58636/0 (socket says 172.21.15.129:58636) 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 -- 172.21.15.129:0/1892414158 learned_addr learned my addr 172.21.15.129:0/1892414158 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe4616e7640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 0x55abcc8a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe460ee6640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc8a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 -- 172.21.15.129:0/1892414158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 msgr2=0x55abcc8a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc8a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 -- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 msgr2=0x55abcc8a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:32.203 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 0x55abcc8a5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 -- 172.21.15.129:0/1892414158 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcc8a7010 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe460ee6640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc8a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe4616e7640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 0x55abcc8a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe461ee8640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe454009830 tx=0x7fe45401e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163f482bd039b931 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.197+0000 7fe444ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45402f070 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fe45401cbc0 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe44c031840 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe454024070 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 10 client.16323 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55abcc8a8320 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe44c031840 con 0x55abcc89c370 2024-09-05T01:24:32.204 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fe454003ce0 con 0x55abcc89c370 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 20 client.16323 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc919790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 4 client.16323 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abcc801d70 con 0x55abcc8ac570 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.198+0000 7fe4616e7640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc919790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.199+0000 7fe4616e7640 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc919790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a65df9a9adee9f56 server_cookie=da84b29811c2d892 in_seq=0 out_seq=0 2024-09-05T01:24:32.205 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.199+0000 7fe444ff9640 10 client.16323 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:32.215 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.214+0000 7fe444ff9640 1 -- 172.21.15.129:0/1892414158 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abcc801d70 con 0x55abcc8ac570 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.214+0000 7fe444ff9640 10 client.16323 handle_command_reply: tid=0 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.214+0000 7fe466322740 1 client.16323 shutdown 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.214+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 msgr2=0x55abcc8a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.214+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe454009830 tx=0x7fe45401e670 comp rx=0 tx=0).stop 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 shutdown_connections 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55abcc8ac570 0x55abcc919790 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55abcc89c370 0x55abcc8a6a00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe444ff9640 0 client.16323 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55abcc8972e0 0x55abcc8a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 --2- 172.21.15.129:0/1892414158 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55abcc8969a0 0x55abcc8a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 >> 172.21.15.129:0/1892414158 conn(0x55abcc892010 msgr2=0x55abcc8923f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 shutdown_connections 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 1 -- 172.21.15.129:0/1892414158 wait complete. 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe466322740 20 client.16323 trim_cache size 0 max 16384 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:24:32.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 msgr2=0x7fe4480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 0x7fe4480a50f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe45401d000 tx=0x7fe454010d90 comp rx=0 tx=0).stop 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 msgr2=0x7fe45c10b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe45800d950 tx=0x7fe45800de10 comp rx=0 tx=0).stop 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 shutdown_connections 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe4480a2c30 0x7fe4480a50f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe45c1025f0 0x7fe45c109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.218 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe45c101c30 0x7fe45c109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 --2- 172.21.15.129:0/2177475600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe45c100a40 0x7fe45c10b3c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 >> 172.21.15.129:0/2177475600 conn(0x7fe45c0fc1e0 msgr2=0x7fe45c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.215+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 shutdown_connections 2024-09-05T01:24:32.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:32.216+0000 7fe46394b640 1 -- 172.21.15.129:0/2177475600 wait complete. 2024-09-05T01:24:33.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:33 smithi183 ceph-mon[34589]: pgmap v628: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T01:24:33.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:33 smithi129 ceph-mon[30979]: pgmap v628: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T01:24:33.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:33 smithi194 ceph-mon[34941]: pgmap v628: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-09-05T01:24:35.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 450.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:35.303 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:35 smithi194 ceph-mon[34941]: pgmap v629: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 574 KiB/s wr, 79 op/s 2024-09-05T01:24:35.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:35 smithi183 ceph-mon[34589]: pgmap v629: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 574 KiB/s wr, 79 op/s 2024-09-05T01:24:35.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:35 smithi129 ceph-mon[30979]: pgmap v629: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 574 KiB/s wr, 79 op/s 2024-09-05T01:24:37.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:37 smithi183 ceph-mon[34589]: pgmap v630: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 288 KiB/s wr, 40 op/s 2024-09-05T01:24:37.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:37 smithi129 ceph-mon[30979]: pgmap v630: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 288 KiB/s wr, 40 op/s 2024-09-05T01:24:37.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:37 smithi194 ceph-mon[34941]: pgmap v630: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 288 KiB/s wr, 40 op/s 2024-09-05T01:24:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:39 smithi183 ceph-mon[34589]: pgmap v631: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 237 KiB/s wr, 40 op/s 2024-09-05T01:24:39.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:39 smithi129 ceph-mon[30979]: pgmap v631: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 237 KiB/s wr, 40 op/s 2024-09-05T01:24:39.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:39 smithi194 ceph-mon[34941]: pgmap v631: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 237 KiB/s wr, 40 op/s 2024-09-05T01:24:40.212 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 455.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:40 smithi183 ceph-mon[34589]: pgmap v632: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-09-05T01:24:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:40 smithi129 ceph-mon[30979]: pgmap v632: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-09-05T01:24:40.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:40 smithi194 ceph-mon[34941]: pgmap v632: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 190 KiB/s wr, 33 op/s 2024-09-05T01:24:43.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:43 smithi183 ceph-mon[34589]: pgmap v633: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-09-05T01:24:43.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:43 smithi129 ceph-mon[30979]: pgmap v633: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-09-05T01:24:43.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:43 smithi194 ceph-mon[34941]: pgmap v633: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 195 KiB/s wr, 34 op/s 2024-09-05T01:24:45.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 460.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- 172.21.15.129:0/989949304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 msgr2=0x7fb9cc1072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/989949304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1072e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009900 tx=0x7fb9c002a410 comp rx=0 tx=0).stop 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- 172.21.15.129:0/989949304 shutdown_connections 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/989949304 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 0x7fb9cc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/989949304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1072e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/989949304 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 0x7fb9cc1049b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- 172.21.15.129:0/989949304 >> 172.21.15.129:0/989949304 conn(0x7fb9cc0fc1e0 msgr2=0x7fb9cc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- 172.21.15.129:0/989949304 shutdown_connections 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- 172.21.15.129:0/989949304 wait complete. 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 Processor -- start 2024-09-05T01:24:45.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.312+0000 7fb9d3aba640 1 -- start start 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 0x7fb9cc197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 0x7fb9cc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10d680 con 0x7fb9cc104ef0 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10d500 con 0x7fb9cc107820 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d3aba640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc10d800 con 0x7fb9cc1025c0 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42484/0 (socket says 172.21.15.129:42484) 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 -- 172.21.15.129:0/1012729814 learned_addr learned my addr 172.21.15.129:0/1012729814 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 0x7fb9cc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1856640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 0x7fb9cc197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 -- 172.21.15.129:0/1012729814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 msgr2=0x7fb9cc197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 0x7fb9cc197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 -- 172.21.15.129:0/1012729814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 msgr2=0x7fb9cc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 0x7fb9cc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 -- 172.21.15.129:0/1012729814 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9cc19f980 con 0x7fb9cc104ef0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9d1055640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009230 tx=0x7fb9c0004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9caffd640 1 -- 172.21.15.129:0/1012729814 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c003f030 con 0x7fb9cc104ef0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.313+0000 7fb9caffd640 1 -- 172.21.15.129:0/1012729814 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c0003cb0 con 0x7fb9cc104ef0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.314+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9cc19fc10 con 0x7fb9cc104ef0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.314+0000 7fb9caffd640 1 -- 172.21.15.129:0/1012729814 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0035070 con 0x7fb9cc104ef0 2024-09-05T01:24:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.314+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9cc1a0150 con 0x7fb9cc104ef0 2024-09-05T01:24:45.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.316+0000 7fb9caffd640 1 -- 172.21.15.129:0/1012729814 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb9c0042030 con 0x7fb9cc104ef0 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.316+0000 7fb9d6855740 1 Processor -- start 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 -- start start 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 0x564acedf1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564aced4d2f0 con 0x564acedea2e0 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564acecf5270 con 0x564acedef370 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d6855740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564aced492b0 con 0x564acede99a0 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9caffd640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 0x7fb9b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42494/0 (socket says 172.21.15.129:42494) 2024-09-05T01:24:45.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 -- 172.21.15.129:0/2278170820 learned_addr learned my addr 172.21.15.129:0/2278170820 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 0x7fb9b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d2057640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9caffd640 1 -- 172.21.15.129:0/1012729814 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb9c00ef9f0 con 0x7fb9cc104ef0 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1055640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 0x564acedf1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 -- 172.21.15.129:0/2278170820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 msgr2=0x564acede9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 -- 172.21.15.129:0/2278170820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 msgr2=0x564acedf1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 0x564acedf1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1856640 1 -- 172.21.15.129:0/2278170820 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acecf4340 con 0x564acedea2e0 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.317+0000 7fb9d1055640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 0x564acedf1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:45.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.318+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 0x7fb9b40a5170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06a980 tx=0x7fb9bc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.318+0000 7fb9d1856640 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400ac20 tx=0x7fb9c400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15728a55d023a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.318+0000 7fb9b3fff640 1 -- 172.21.15.129:0/2278170820 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c401d070 con 0x564acedea2e0 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.318+0000 7fb9b3fff640 1 -- 172.21.15.129:0/2278170820 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c4018af0 con 0x564acedea2e0 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.318+0000 7fb9b3fff640 1 -- 172.21.15.129:0/2278170820 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4010a50 con 0x564acedea2e0 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- 172.21.15.129:0/2278170820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 msgr2=0x564acedeeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400ac20 tx=0x7fb9c400f790 comp rx=0 tx=0).stop 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- 172.21.15.129:0/2278170820 shutdown_connections 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedef370 0x564acedf1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedeeda0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 --2- 172.21.15.129:0/2278170820 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- 172.21.15.129:0/2278170820 >> 172.21.15.129:0/2278170820 conn(0x564acede5010 msgr2=0x564acede5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- 172.21.15.129:0/2278170820 shutdown_connections 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- 172.21.15.129:0/2278170820 wait complete. 2024-09-05T01:24:45.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 -- start start 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.319+0000 7fb9d6855740 1 Processor -- start 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acedf3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acedfb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564acedf2340 con 0x564acedea2e0 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564acecd6290 con 0x564acede99a0 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564acedf24c0 con 0x564acedef370 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d6855740 10 client.0 _ll_register_callbacks cb 0x564aced07140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42502/0 (socket says 172.21.15.129:42502) 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 -- 172.21.15.129:0/4188049247 learned_addr learned my addr 172.21.15.129:0/4188049247 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d2057640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acedfb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1856640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acedf3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 -- 172.21.15.129:0/4188049247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 msgr2=0x564acedfb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acedfb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 -- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 msgr2=0x564acedf3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acedf3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 -- 172.21.15.129:0/4188049247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acea60210 con 0x564acedea2e0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d2057640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acedfb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9d1055640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b8350 tx=0x7fb9c0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300595aabd343757 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.320+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00be070 con 0x564acedea2e0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb9c00b8560 con 0x564acedea2e0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c4018af0 con 0x564acedea2e0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0049040 con 0x564acedea2e0 2024-09-05T01:24:45.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 10 client.16335 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564aceb2c620 con 0x564acedea2e0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9c4018af0 con 0x564acedea2e0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c003b4f0 con 0x564acedea2e0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 10 client.16335 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 10 client.16335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 20 client.16335 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acedff7f0 0x564acee01c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 4 client.16335 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aced5aa20 con 0x564acedff7f0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.321+0000 7fb9d1856640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acedff7f0 0x564acee01c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.322+0000 7fb9b17fa640 10 client.16335 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.322+0000 7fb9d1856640 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acedff7f0 0x564acee01c70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd0bf68c64e57c server_cookie=15d4f9d74b69d6d7 in_seq=0 out_seq=0 2024-09-05T01:24:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9b17fa640 1 -- 172.21.15.129:0/4188049247 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564aced5aa20 con 0x564acedff7f0 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9b17fa640 10 client.16335 handle_command_reply: tid=0 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 client.16335 shutdown 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 msgr2=0x564acedf4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b8350 tx=0x7fb9c0009260 comp rx=0 tx=0).stop 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 shutdown_connections 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acedff7f0 0x564acee01c70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acedfb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acedf4130 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9b17fa640 0 client.16335 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 --2- 172.21.15.129:0/4188049247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acedf3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 >> 172.21.15.129:0/4188049247 conn(0x564acede5010 msgr2=0x564acede6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 shutdown_connections 2024-09-05T01:24:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 1 -- 172.21.15.129:0/4188049247 wait complete. 2024-09-05T01:24:45.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.323+0000 7fb9d6855740 20 client.16335 trim_cache size 0 max 16384 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 Processor -- start 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 -- start start 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedea2e0 0x564acee06570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1ad0e0 con 0x564acedef370 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1aa9b0 con 0x564acedea2e0 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d6855740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1a8800 con 0x564acede99a0 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.337+0000 7fb9d2057640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42504/0 (socket says 172.21.15.129:42504) 2024-09-05T01:24:45.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 -- 172.21.15.129:0/1343637587 learned_addr learned my addr 172.21.15.129:0/1343637587 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d1055640 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedea2e0 0x564acee06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d1856640 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 -- 172.21.15.129:0/1343637587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 msgr2=0x564acede40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 -- 172.21.15.129:0/1343637587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedea2e0 msgr2=0x564acee06570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedea2e0 0x564acee06570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 -- 172.21.15.129:0/1343637587 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acedeab30 con 0x564acedef370 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9d2057640 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009b10 tx=0x7fb9bc00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55ce05f93ad150a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9b37fe640 1 -- 172.21.15.129:0/1343637587 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc010ba0 con 0x564acedef370 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9b37fe640 1 -- 172.21.15.129:0/1343637587 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc010d40 con 0x564acedef370 2024-09-05T01:24:45.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.338+0000 7fb9b37fe640 1 -- 172.21.15.129:0/1343637587 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc020680 con 0x564acedef370 2024-09-05T01:24:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- 172.21.15.129:0/1343637587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 msgr2=0x564acee06ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009b10 tx=0x7fb9bc00c000 comp rx=0 tx=0).stop 2024-09-05T01:24:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- 172.21.15.129:0/1343637587 shutdown_connections 2024-09-05T01:24:45.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedef370 0x564acee06ab0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acedea2e0 0x564acee06570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 --2- 172.21.15.129:0/1343637587 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acede99a0 0x564acede40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- 172.21.15.129:0/1343637587 >> 172.21.15.129:0/1343637587 conn(0x564acede5010 msgr2=0x564acede39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- 172.21.15.129:0/1343637587 shutdown_connections 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- 172.21.15.129:0/1343637587 wait complete. 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 -- start start 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 Processor -- start 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.339+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acee65f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acee6d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1a8e70 con 0x564acedea2e0 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1aa2e0 con 0x564acede99a0 2024-09-05T01:24:45.344 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb9cc1a9540 con 0x564acedef370 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 10 client.0 _ll_register_callbacks cb 0x7fb9c0005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42516/0 (socket says 172.21.15.129:42516) 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 -- 172.21.15.129:0/3326999211 learned_addr learned my addr 172.21.15.129:0/3326999211 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d2057640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acee6d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1856640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acee65f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 -- 172.21.15.129:0/3326999211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 msgr2=0x564acee6d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acee6d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 -- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 msgr2=0x564acee65f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acee65f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 -- 172.21.15.129:0/3326999211 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acee6db00 con 0x564acedea2e0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d1055640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b88c0 tx=0x7fb9c003bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d459c40a514774d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:45.345 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00be040 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc010d40 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7fb9c0003800 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0003e60 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 10 client.16341 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.340+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564acee6ee80 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9bc010d40 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7fb9c00a5080 con 0x564acedea2e0 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 20 client.16341 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acee6f620 0x564acee71a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 4 client.16341 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:24:45.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aced54d70 con 0x564acee6f620 2024-09-05T01:24:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d1856640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acee6f620 0x564acee71a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9d1856640 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acee6f620 0x564acee71a40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a225a5c622b8f0bf server_cookie=c34bf76c53952208 in_seq=0 out_seq=0 2024-09-05T01:24:45.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.341+0000 7fb9b17fa640 10 client.16341 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9b17fa640 1 -- 172.21.15.129:0/3326999211 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564aced54d70 con 0x564acee6f620 2024-09-05T01:24:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9b17fa640 10 client.16341 handle_command_reply: tid=0 2024-09-05T01:24:45.353 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 client.16341 shutdown 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 msgr2=0x564acee664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00b88c0 tx=0x7fb9c003bf80 comp rx=0 tx=0).stop 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 shutdown_connections 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564acee6f620 0x564acee71a40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564acedef370 0x564acee6d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564acedea2e0 0x564acee664b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9b17fa640 0 client.16341 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 --2- 172.21.15.129:0/3326999211 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564acede99a0 0x564acee65f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 >> 172.21.15.129:0/3326999211 conn(0x564acede5010 msgr2=0x564acede39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 shutdown_connections 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.352+0000 7fb9d6855740 1 -- 172.21.15.129:0/3326999211 wait complete. 2024-09-05T01:24:45.354 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d6855740 20 client.16341 trim_cache size 0 max 16384 2024-09-05T01:24:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:24:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:24:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:24:45.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 msgr2=0x7fb9b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 0x7fb9b40a5170 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06a980 tx=0x7fb9bc004ab0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 msgr2=0x7fb9cc1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009230 tx=0x7fb9c0004340 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 shutdown_connections 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb9b40a2cb0 0x7fb9b40a5170 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb9cc107820 0x7fb9cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb9cc104ef0 0x7fb9cc1982a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 --2- 172.21.15.129:0/1012729814 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb9cc1025c0 0x7fb9cc197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:45.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 >> 172.21.15.129:0/1012729814 conn(0x7fb9cc0fc1e0 msgr2=0x7fb9cc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 shutdown_connections 2024-09-05T01:24:45.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:45.353+0000 7fb9d3aba640 1 -- 172.21.15.129:0/1012729814 wait complete. 2024-09-05T01:24:45.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:45 smithi183 ceph-mon[34589]: pgmap v634: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 205 KiB/s wr, 35 op/s 2024-09-05T01:24:45.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:45 smithi129 ceph-mon[30979]: pgmap v634: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 205 KiB/s wr, 35 op/s 2024-09-05T01:24:45.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:45 smithi194 ceph-mon[34941]: pgmap v634: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 205 KiB/s wr, 35 op/s 2024-09-05T01:24:46.105 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:24:46.367 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:47.338 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:47 smithi129 ceph-mon[30979]: pgmap v635: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 197 KiB/s wr, 32 op/s 2024-09-05T01:24:47.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:47 smithi183 ceph-mon[34589]: pgmap v635: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 197 KiB/s wr, 32 op/s 2024-09-05T01:24:47.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:47 smithi194 ceph-mon[34941]: pgmap v635: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 197 KiB/s wr, 32 op/s 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 -- 172.21.15.129:0/3979069802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 msgr2=0x7f35b01067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 --2- 172.21.15.129:0/3979069802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b01067c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f3598009900 tx=0x7f359802a410 comp rx=0 tx=0).stop 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 -- 172.21.15.129:0/3979069802 shutdown_connections 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 --2- 172.21.15.129:0/3979069802 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0106ef0 0x7f35b010ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 --2- 172.21.15.129:0/3979069802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b01067c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 --2- 172.21.15.129:0/3979069802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0101dc0 0x7f35b01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 -- 172.21.15.129:0/3979069802 >> 172.21.15.129:0/3979069802 conn(0x7f35b00fd880 msgr2=0x7f35b00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 -- 172.21.15.129:0/3979069802 shutdown_connections 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.561+0000 7f35b5640640 1 -- 172.21.15.129:0/3979069802 wait complete. 2024-09-05T01:24:47.587 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 Processor -- start 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 -- start start 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0101dc0 0x7f35b00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0106ef0 0x7f35b0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0111e00 con 0x7f35b0102770 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0111c80 con 0x7f35b0106ef0 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35b5640640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0111f80 con 0x7f35b0101dc0 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42536/0 (socket says 172.21.15.129:42536) 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 -- 172.21.15.129:0/3622916400 learned_addr learned my addr 172.21.15.129:0/3622916400 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35aeffd640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0101dc0 0x7f35b00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 -- 172.21.15.129:0/3622916400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0101dc0 msgr2=0x7f35b00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:47.588 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0101dc0 0x7f35b00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35af7fe640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0106ef0 0x7f35b0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 -- 172.21.15.129:0/3622916400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0106ef0 msgr2=0x7f35b0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0106ef0 0x7f35b0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 -- 172.21.15.129:0/3622916400 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b01a6830 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.562+0000 7f35a7fff640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3598006010 tx=0x7f3598004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.563+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359803f030 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.563+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01a6ac0 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.563+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35980035f0 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.563+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3598035070 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.563+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a6fa0 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.564+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b01021a0 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.565+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3598003b10 con 0x7f35b0102770 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.566+0000 7f35acff9640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 0x7f359c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:47.589 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.566+0000 7f35aeffd640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 0x7f359c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:47.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.566+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f35980f0290 con 0x7f35b0102770 2024-09-05T01:24:47.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.566+0000 7f35aeffd640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 0x7f359c0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35a00058c0 tx=0x7f35a000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:47.590 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.569+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35980bcde0 con 0x7f35b0102770 2024-09-05T01:24:47.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.776+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35b010a130 con 0x7f35b0102770 2024-09-05T01:24:47.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.777+0000 7f35acff9640 1 -- 172.21.15.129:0/3622916400 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11933 (secure 0 0 0) 0x7f35980bc6e0 con 0x7f35b0102770 2024-09-05T01:24:47.778 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:24:47.778 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":30,"btime":"2024-09-05T01:18:34:938975+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:18:34.938955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25198":{"gid":25198,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/2216480546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":2216480546},{"type":"v1","addr":"172.21.15.194:6837","nonce":2216480546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:24:47.778 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 30 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 msgr2=0x7f359c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 0x7f359c0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f35a00058c0 tx=0x7f35a000a4c0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 msgr2=0x7f35b0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3598006010 tx=0x7f3598004270 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 shutdown_connections 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f359c0a2b80 0x7f359c0a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f35b0106ef0 0x7f35b0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f35b0102770 0x7f35b0074a30 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 --2- 172.21.15.129:0/3622916400 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f35b0101dc0 0x7f35b00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 >> 172.21.15.129:0/3622916400 conn(0x7f35b00fd880 msgr2=0x7f35b00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:47.780 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 shutdown_connections 2024-09-05T01:24:47.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:47.779+0000 7f35b5640640 1 -- 172.21.15.129:0/3622916400 wait complete. 2024-09-05T01:24:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:48 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3622916400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:48.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3622916400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:48 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3622916400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:24:48.548 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id ba18b76 --release --await' 2024-09-05T01:24:48.549 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id ba18b76 --release --await 2024-09-05T01:24:48.811 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:24:49.231 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:49 smithi129 ceph-mon[30979]: pgmap v636: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 218 KiB/s wr, 37 op/s 2024-09-05T01:24:49.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 -- 172.21.15.129:0/670058650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f454410e440 msgr2=0x7f4544110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 --2- 172.21.15.129:0/670058650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f454410e440 0x7f4544110830 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f453c009900 tx=0x7f453c02a480 comp rx=0 tx=0).stop 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 -- 172.21.15.129:0/670058650 shutdown_connections 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 --2- 172.21.15.129:0/670058650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f454410e440 0x7f4544110830 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 --2- 172.21.15.129:0/670058650 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4544074e30 0x7f454410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 --2- 172.21.15.129:0/670058650 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f45440748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 -- 172.21.15.129:0/670058650 >> 172.21.15.129:0/670058650 conn(0x7f45440fd8c0 msgr2=0x7f45440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 -- 172.21.15.129:0/670058650 shutdown_connections 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.319+0000 7f454b546640 1 -- 172.21.15.129:0/670058650 wait complete. 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 Processor -- start 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 -- start start 2024-09-05T01:24:49.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f454419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 0x7f454419ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4544114010 con 0x7f4544074e30 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4544113e90 con 0x7f454410e440 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f454b546640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4544114190 con 0x7f4544074510 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f4548ae1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f4548ae1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42546/0 (socket says 172.21.15.129:42546) 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.320+0000 7f4548ae1640 1 -- 172.21.15.129:0/3460902599 learned_addr learned my addr 172.21.15.129:0/3460902599 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 0x7f454419ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f45492e2640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f454419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 -- 172.21.15.129:0/3460902599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 msgr2=0x7f454419a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f454419a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 -- 172.21.15.129:0/3460902599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 msgr2=0x7f454419ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 0x7f454419ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 -- 172.21.15.129:0/3460902599 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454419f4e0 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f45492e2640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f454419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 0x7f454419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f453a7fc640 1 -- 172.21.15.129:0/3460902599 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534017070 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f453a7fc640 1 -- 172.21.15.129:0/3460902599 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4534009c30 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f453a7fc640 1 -- 172.21.15.129:0/3460902599 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453400f620 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454419f7d0 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.321+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45441a70b0 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.323+0000 7f453a7fc640 1 -- 172.21.15.129:0/3460902599 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f45340109a0 con 0x7f4544074e30 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 Processor -- start 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 -- start start 2024-09-05T01:24:49.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f453a7fc640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce676d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce679630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5597ce5d1620 con 0x5597ce66c060 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5597ce578e50 con 0x5597ce66c980 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f454d91f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5597ce5cd5e0 con 0x5597ce677240 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f453a7fc640 1 -- 172.21.15.129:0/3460902599 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f45340cf090 con 0x7f4544074e30 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42552/0 (socket says 172.21.15.129:42552) 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f45492e2640 1 -- 172.21.15.129:0/4267446613 learned_addr learned my addr 172.21.15.129:0/4267446613 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f45492e2640 1 -- 172.21.15.129:0/4267446613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 msgr2=0x5597ce679630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.324+0000 7f45492e2640 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce679630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f45492e2640 1 -- 172.21.15.129:0/4267446613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 msgr2=0x5597ce676d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f45492e2640 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce676d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f45492e2640 1 -- 172.21.15.129:0/4267446613 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ce569d40 con 0x5597ce66c060 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f45492e2640 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f454000af40 tx=0x7f454000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d552a372f390ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 0x7f45300a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f453c031440 tx=0x7f453c009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f452f7fe640 1 -- 172.21.15.129:0/4267446613 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540004d20 con 0x5597ce66c060 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f452f7fe640 1 -- 172.21.15.129:0/4267446613 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4540011b00 con 0x5597ce66c060 2024-09-05T01:24:49.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.325+0000 7f452f7fe640 1 -- 172.21.15.129:0/4267446613 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45400055e0 con 0x5597ce66c060 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- 172.21.15.129:0/4267446613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 msgr2=0x5597ce66c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f454000af40 tx=0x7f454000f7f0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- 172.21.15.129:0/4267446613 shutdown_connections 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce679630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce676d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- 172.21.15.129:0/4267446613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce66c440 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- 172.21.15.129:0/4267446613 >> 172.21.15.129:0/4267446613 conn(0x5597ce6685f0 msgr2=0x5597ce6689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- 172.21.15.129:0/4267446613 shutdown_connections 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- 172.21.15.129:0/4267446613 wait complete. 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 -- start start 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 Processor -- start 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.326+0000 7f454d91f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce677240 0x5597ce674bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5597ce679e10 con 0x5597ce66c060 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5597ce559f10 con 0x5597ce677240 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5597ce679f90 con 0x5597ce66c980 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 10 client.0 _ll_register_callbacks cb 0x5597ce57ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42568/0 (socket says 172.21.15.129:42568) 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 -- 172.21.15.129:0/458759875 learned_addr learned my addr 172.21.15.129:0/458759875 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f4548ae1640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 -- 172.21.15.129:0/458759875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 msgr2=0x5597ce674680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 -- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce677240 msgr2=0x5597ce674bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce677240 0x5597ce674bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 -- 172.21.15.129:0/458759875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ce2e0580 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f45492e2640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45400119c0 tx=0x7f45400119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abda31cfd4929398 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f452cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540029060 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4540004e10 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4540011b00 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4540030050 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 10 client.16359 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597ce3aa960 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.327+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4540011b00 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f45400197d0 con 0x5597ce66c060 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 10 client.16359 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:49.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 10 client.16359 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 20 client.16359 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce686dc0 0x5597ce6891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 4 client.16359 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597ce5deb90 con 0x5597ce686dc0 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f4548ae1640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce686dc0 0x5597ce6891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f452cff9640 10 client.16359 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.328+0000 7f4548ae1640 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce686dc0 0x5597ce6891b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eb08af7f625d74a server_cookie=6afce2b0551b8e82 in_seq=0 out_seq=0 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f452cff9640 1 -- 172.21.15.129:0/458759875 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597ce5deb90 con 0x5597ce686dc0 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f452cff9640 10 client.16359 handle_command_reply: tid=0 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 client.16359 shutdown 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 msgr2=0x5597ce674140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f45400119c0 tx=0x7f45400119f0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 shutdown_connections 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce686dc0 0x5597ce6891b0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce677240 0x5597ce674bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 --2- 172.21.15.129:0/458759875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce674140 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f452cff9640 0 client.16359 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 >> 172.21.15.129:0/458759875 conn(0x5597ce6685f0 msgr2=0x5597ce66e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 shutdown_connections 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 1 -- 172.21.15.129:0/458759875 wait complete. 2024-09-05T01:24:49.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.329+0000 7f454d91f740 20 client.16359 trim_cache size 0 max 16384 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.346+0000 7f454d91f740 1 Processor -- start 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.346+0000 7f454d91f740 1 -- start start 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c060 0x5597ce66eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b38a0 con 0x5597ce677240 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b1170 con 0x5597ce66c060 2024-09-05T01:24:49.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f454d91f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45441aefc0 con 0x5597ce66c980 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42580/0 (socket says 172.21.15.129:42580) 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 -- 172.21.15.129:0/1392251288 learned_addr learned my addr 172.21.15.129:0/1392251288 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f45492e2640 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c060 0x5597ce66eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4548ae1640 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 -- 172.21.15.129:0/1392251288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 msgr2=0x5597ce674810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 -- 172.21.15.129:0/1392251288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c060 msgr2=0x5597ce66eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c060 0x5597ce66eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 -- 172.21.15.129:0/1392251288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597ce679990 con 0x5597ce677240 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f4549ae3640 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f45440754f0 tx=0x7f453c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fff32dc56cbebb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f452effd640 1 -- 172.21.15.129:0/1392251288 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c04f070 con 0x5597ce677240 2024-09-05T01:24:49.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f452effd640 1 -- 172.21.15.129:0/1392251288 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f453c043070 con 0x5597ce677240 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.347+0000 7f452effd640 1 -- 172.21.15.129:0/1392251288 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c035050 con 0x5597ce677240 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- 172.21.15.129:0/1392251288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 msgr2=0x5597ce674d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f45440754f0 tx=0x7f453c02a480 comp rx=0 tx=0).stop 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- 172.21.15.129:0/1392251288 shutdown_connections 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce677240 0x5597ce674d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce66c980 0x5597ce674810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 --2- 172.21.15.129:0/1392251288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c060 0x5597ce66eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- 172.21.15.129:0/1392251288 >> 172.21.15.129:0/1392251288 conn(0x5597ce6685f0 msgr2=0x5597ce674100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- 172.21.15.129:0/1392251288 shutdown_connections 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- 172.21.15.129:0/1392251288 wait complete. 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 -- start start 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.348+0000 7f454d91f740 1 Processor -- start 2024-09-05T01:24:49.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce67b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce67b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45441af630 con 0x5597ce66c060 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45441b0aa0 con 0x5597ce66c980 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45441afd00 con 0x5597ce677240 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f454d91f740 10 client.0 _ll_register_callbacks cb 0x7f4540011650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42596/0 (socket says 172.21.15.129:42596) 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 -- 172.21.15.129:0/3247119978 learned_addr learned my addr 172.21.15.129:0/3247119978 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce67b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 -- 172.21.15.129:0/3247119978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 msgr2=0x5597ce67b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce67b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce67b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 -- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 msgr2=0x5597ce67b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce67b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f45492e2640 1 -- 172.21.15.129:0/3247119978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597ce6f07b0 con 0x5597ce66c060 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce67b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.349+0000 7f4549ae3640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce67b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f45492e2640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f454000b010 tx=0x7f454000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d95c6d232134f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540029040 con 0x5597ce66c060 2024-09-05T01:24:49.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f4540021070 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f453c043070 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4540019640 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 10 client.16365 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597ce6f1a50 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f453c043070 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f45400042e0 con 0x5597ce66c060 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 10 client.16365 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 10 client.16365 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 20 client.16365 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:49.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce6f2160 0x5597ce6f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:49.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 4 client.16365 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ba18b76", "release": true, "await": true}] 2024-09-05T01:24:49.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.350+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ba18b76", "release": true, "await": true}) -- 0x5597ce5d90a0 con 0x5597ce6f2160 2024-09-05T01:24:49.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.351+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce6f2160 0x5597ce6f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:49.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.351+0000 7f452cff9640 10 client.16365 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:49.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.351+0000 7f4548ae1640 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce6f2160 0x5597ce6f4580 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286276590f066d73 server_cookie=17789dc6819a889b in_seq=0 out_seq=0 2024-09-05T01:24:49.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:49 smithi183 ceph-mon[34589]: pgmap v636: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 218 KiB/s wr, 37 op/s 2024-09-05T01:24:49.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.424+0000 7f452cff9640 1 -- 172.21.15.129:0/3247119978 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5597ce5d90a0 con 0x5597ce6f2160 2024-09-05T01:24:49.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.425+0000 7f452cff9640 10 client.16365 handle_command_reply: tid=0 2024-09-05T01:24:49.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.425+0000 7f454d91f740 1 client.16365 shutdown 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 msgr2=0x5597ce67aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f454000b010 tx=0x7f454000a7b0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 shutdown_connections 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5597ce6f2160 0x5597ce6f4580 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5597ce677240 0x5597ce67b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5597ce66c980 0x5597ce67b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f452cff9640 0 client.16365 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 --2- 172.21.15.129:0/3247119978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5597ce66c060 0x5597ce67aec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.426+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 >> 172.21.15.129:0/3247119978 conn(0x5597ce6685f0 msgr2=0x5597ce674890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.427+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 shutdown_connections 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.427+0000 7f454d91f740 1 -- 172.21.15.129:0/3247119978 wait complete. 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.427+0000 7f454d91f740 20 client.16365 trim_cache size 0 max 16384 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 30, 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:24:49.429 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 36, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "ba18b76": { 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 36, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:24:49.430 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:49.431 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:49.432 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:24:49.432 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.428+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 msgr2=0x7f45300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.428+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 0x7f45300a50c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f453c031440 tx=0x7f453c009650 comp rx=0 tx=0).stop 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.428+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 msgr2=0x7f454419aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.428+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f453400d950 tx=0x7f453400de10 comp rx=0 tx=0).stop 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 shutdown_connections 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f454410e440 0x7f454419ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4544074e30 0x7f454419aa70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 --2- 172.21.15.129:0/3460902599 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4544074510 0x7f454419a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:49.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 >> 172.21.15.129:0/3460902599 conn(0x7f45440fd8c0 msgr2=0x7f45440fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:49.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 shutdown_connections 2024-09-05T01:24:49.434 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:24:49.429+0000 7f454b546640 1 -- 172.21.15.129:0/3460902599 wait complete. 2024-09-05T01:24:49.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:49 smithi194 ceph-mon[34941]: pgmap v636: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 218 KiB/s wr, 37 op/s 2024-09-05T01:24:50.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 465.0 s, 173.6 tps, lat 3520.341 ms stddev 9526.226 2024-09-05T01:24:50.471 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ba18b76', total seconds elapsed: 34.3 2024-09-05T01:24:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:51 smithi183 ceph-mon[34589]: pgmap v637: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-09-05T01:24:51.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:51 smithi129 ceph-mon[30979]: pgmap v637: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-09-05T01:24:51.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:51 smithi194 ceph-mon[34941]: pgmap v637: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 173 KiB/s wr, 28 op/s 2024-09-05T01:24:53.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:53 smithi183 ceph-mon[34589]: pgmap v638: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 665 KiB/s wr, 76 op/s 2024-09-05T01:24:53.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:53 smithi129 ceph-mon[30979]: pgmap v638: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 665 KiB/s wr, 76 op/s 2024-09-05T01:24:53.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:53 smithi194 ceph-mon[34941]: pgmap v638: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 665 KiB/s wr, 76 op/s 2024-09-05T01:24:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:54 smithi183 ceph-mon[34589]: pgmap v639: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-05T01:24:54.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:54 smithi129 ceph-mon[30979]: pgmap v639: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-05T01:24:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:54 smithi194 ceph-mon[34941]: pgmap v639: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-05T01:24:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 470.0 s, 1040.6 tps, lat 97.273 ms stddev 35.664 2024-09-05T01:24:57.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:57 smithi183 ceph-mon[34589]: pgmap v640: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-05T01:24:57.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:57 smithi129 ceph-mon[30979]: pgmap v640: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-05T01:24:57.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:57 smithi194 ceph-mon[34941]: pgmap v640: 129 pgs: 129 active+clean; 1.3 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.4 MiB/s wr, 262 op/s 2024-09-05T01:24:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 -- 172.21.15.129:0/1189041117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410df00 msgr2=0x7f56d4114790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 --2- 172.21.15.129:0/1189041117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410df00 0x7f56d4114790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009930 tx=0x7f56c402a420 comp rx=0 tx=0).stop 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 -- 172.21.15.129:0/1189041117 shutdown_connections 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 --2- 172.21.15.129:0/1189041117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410df00 0x7f56d4114790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 --2- 172.21.15.129:0/1189041117 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410d540 0x7f56d410d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 --2- 172.21.15.129:0/1189041117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410c6b0 0x7f56d410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.501+0000 7f56d3fff640 1 -- 172.21.15.129:0/1189041117 >> 172.21.15.129:0/1189041117 conn(0x7f56d406bda0 msgr2=0x7f56d406c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.504+0000 7f56d3fff640 1 -- 172.21.15.129:0/1189041117 shutdown_connections 2024-09-05T01:24:58.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.504+0000 7f56d3fff640 1 -- 172.21.15.129:0/1189041117 wait complete. 2024-09-05T01:24:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.505+0000 7f56d3fff640 1 Processor -- start 2024-09-05T01:24:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.506+0000 7f56d3fff640 1 -- start start 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.506+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410d540 0x7f56d41a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410df00 0x7f56d41a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:33866/0 (socket says 172.21.15.129:33866) 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d2ffd640 1 -- 172.21.15.129:0/2200504906 learned_addr learned my addr 172.21.15.129:0/2200504906 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:58.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4117ec0 con 0x7f56d410d540 2024-09-05T01:24:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4117d40 con 0x7f56d410c6b0 2024-09-05T01:24:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d37fe640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410df00 0x7f56d41a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d27fc640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410d540 0x7f56d41a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.507+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56d4118040 con 0x7f56d410df00 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 -- 172.21.15.129:0/2200504906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410df00 msgr2=0x7f56d41a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410df00 0x7f56d41a9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 -- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410d540 msgr2=0x7f56d41a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410d540 0x7f56d41a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d41aa180 con 0x7f56d410c6b0 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56c800d770 tx=0x7f56c800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.508+0000 7f56bbfff640 1 -- 172.21.15.129:0/2200504906 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8017070 con 0x7f56d410c6b0 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.509+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d41aa470 con 0x7f56d410c6b0 2024-09-05T01:24:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.509+0000 7f56bbfff640 1 -- 172.21.15.129:0/2200504906 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56c8009c30 con 0x7f56d410c6b0 2024-09-05T01:24:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.509+0000 7f56bbfff640 1 -- 172.21.15.129:0/2200504906 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8003c50 con 0x7f56d410c6b0 2024-09-05T01:24:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.509+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d41aaa10 con 0x7f56d410c6b0 2024-09-05T01:24:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.512+0000 7f56bbfff640 1 -- 172.21.15.129:0/2200504906 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f56c8003df0 con 0x7f56d410c6b0 2024-09-05T01:24:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.512+0000 7f56dc02f740 1 Processor -- start 2024-09-05T01:24:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.513+0000 7f56dc02f740 1 -- start start 2024-09-05T01:24:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.513+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.513+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff434b0 0x5637eff47f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.513+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56dc02f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637efea62f0 con 0x5637eff434b0 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56dc02f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5637efe4e270 con 0x5637eff48540 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56dc02f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5637efea22b0 con 0x5637eff42b70 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff434b0 0x5637eff47f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36034/0 (socket says 172.21.15.129:36034) 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 -- 172.21.15.129:0/1896645955 learned_addr learned my addr 172.21.15.129:0/1896645955 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d27fc640 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56bbfff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 0x7f56bc0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 -- 172.21.15.129:0/1896645955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 msgr2=0x5637eff4aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff4aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 -- 172.21.15.129:0/1896645955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff434b0 msgr2=0x5637eff47f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff434b0 0x5637eff47f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 -- 172.21.15.129:0/1896645955 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637efe4d340 con 0x5637eff42b70 2024-09-05T01:24:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56bbfff640 1 -- 172.21.15.129:0/2200504906 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f56c80cee20 con 0x7f56d410c6b0 2024-09-05T01:24:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.514+0000 7f56d37fe640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 0x7f56bc0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.515+0000 7f56d37fe640 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56c402a990 tx=0x7f56c40042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3bf014132dead98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.515+0000 7f56b97fa640 1 -- 172.21.15.129:0/1896645955 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4035030 con 0x5637eff42b70 2024-09-05T01:24:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.515+0000 7f56b97fa640 1 -- 172.21.15.129:0/1896645955 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56c40054a0 con 0x5637eff42b70 2024-09-05T01:24:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56b97fa640 1 -- 172.21.15.129:0/1896645955 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4003bf0 con 0x5637eff42b70 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 -- 172.21.15.129:0/1896645955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 msgr2=0x5637eff42f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56d37fe640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 0x7f56bc0a5180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005ab0 tx=0x7f56c4003ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f56c402a990 tx=0x7f56c40042c0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 -- 172.21.15.129:0/1896645955 shutdown_connections 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff4aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff434b0 0x5637eff47f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 --2- 172.21.15.129:0/1896645955 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff42b70 0x5637eff42f70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 -- 172.21.15.129:0/1896645955 >> 172.21.15.129:0/1896645955 conn(0x5637eff3e1e0 msgr2=0x5637eff3e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.516+0000 7f56dc02f740 1 -- 172.21.15.129:0/1896645955 shutdown_connections 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 -- 172.21.15.129:0/1896645955 wait complete. 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 -- start start 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 Processor -- start 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff4cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff4d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.517+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff542c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56dc02f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5637eff4b500 con 0x5637eff42b70 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56dc02f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5637efe2f290 con 0x5637eff48540 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56dc02f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5637eff4b680 con 0x5637eff434b0 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff4cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56dc02f740 10 client.0 _ll_register_callbacks cb 0x5637efe60140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff4cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51674/0 (socket says 172.21.15.129:51674) 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d2ffd640 1 -- 172.21.15.129:0/4094389479 learned_addr learned my addr 172.21.15.129:0/4094389479 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d27fc640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff4d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 -- 172.21.15.129:0/4094389479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 msgr2=0x5637eff4d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff4d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 -- 172.21.15.129:0/4094389479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 msgr2=0x5637eff4cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff4cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.518+0000 7f56d37fe640 1 -- 172.21.15.129:0/4094389479 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637efbb9210 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56d37fe640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff542c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009900 tx=0x7f56c4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5584b4b5a10d1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4035040 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f56c4003bb0 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56c40054a0 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56c4065050 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56dc02f740 10 client.25916 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.519+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637efc85620 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56c4031c70 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f56c40047c0 con 0x5637eff48540 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 10 client.25916 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 20 client.25916 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637eff58b60 0x5637eff5af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 4 client.25916 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56c40054a0 con 0x5637eff58b60 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.520+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637eff58b60 0x5637eff5af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.521+0000 7f56b6ff5640 10 client.25916 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.521+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637eff58b60 0x5637eff5af50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b4eabf606bef18c server_cookie=2a3c9626b659c940 in_seq=0 out_seq=0 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.521+0000 7f56b6ff5640 1 -- 172.21.15.129:0/4094389479 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f56c40054a0 con 0x5637eff58b60 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.521+0000 7f56b6ff5640 10 client.25916 handle_command_reply: tid=0 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.521+0000 7f56dc02f740 1 client.25916 shutdown 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 msgr2=0x5637eff542c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff542c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009900 tx=0x7f56c4040000 comp rx=0 tx=0).stop 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 shutdown_connections 2024-09-05T01:24:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637eff58b60 0x5637eff5af50 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff542c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff4d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 --2- 172.21.15.129:0/4094389479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff4cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56b6ff5640 0 client.25916 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 >> 172.21.15.129:0/4094389479 conn(0x5637eff3e1e0 msgr2=0x5637eff3f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 shutdown_connections 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 1 -- 172.21.15.129:0/4094389479 wait complete. 2024-09-05T01:24:58.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.522+0000 7f56dc02f740 20 client.25916 trim_cache size 0 max 16384 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.547+0000 7f56dc02f740 1 Processor -- start 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 -- start start 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff54280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff3cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b7920 con 0x5637eff42b70 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b51f0 con 0x5637eff48540 2024-09-05T01:24:58.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.548+0000 7f56dc02f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b3040 con 0x5637eff434b0 2024-09-05T01:24:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.549+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.549+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51690/0 (socket says 172.21.15.129:51690) 2024-09-05T01:24:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.549+0000 7f56d2ffd640 1 -- 172.21.15.129:0/964924414 learned_addr learned my addr 172.21.15.129:0/964924414 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.549+0000 7f56d2ffd640 1 -- 172.21.15.129:0/964924414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 msgr2=0x5637eff54280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:24:58.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.549+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff54280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.550+0000 7f56d2ffd640 1 -- 172.21.15.129:0/964924414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 msgr2=0x5637eff3cfd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.550+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff3cfd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.550+0000 7f56d2ffd640 1 -- 172.21.15.129:0/964924414 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637eff3e000 con 0x5637eff42b70 2024-09-05T01:24:58.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.550+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f56c809af10 tx=0x7f56c80ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dc0832a1cf8d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.551+0000 7f56b8ff9640 1 -- 172.21.15.129:0/964924414 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c8020040 con 0x5637eff42b70 2024-09-05T01:24:58.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.551+0000 7f56b8ff9640 1 -- 172.21.15.129:0/964924414 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56c80b9e50 con 0x5637eff42b70 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 -- 172.21.15.129:0/964924414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 msgr2=0x5637eff3ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f56c809af10 tx=0x7f56c80ce040 comp rx=0 tx=0).stop 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 -- 172.21.15.129:0/964924414 shutdown_connections 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff48540 0x5637eff3cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff434b0 0x5637eff54280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 --2- 172.21.15.129:0/964924414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff42b70 0x5637eff3ca90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.552+0000 7f56dc02f740 1 -- 172.21.15.129:0/964924414 >> 172.21.15.129:0/964924414 conn(0x5637eff3e1e0 msgr2=0x5637eff4cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.553+0000 7f56dc02f740 1 -- 172.21.15.129:0/964924414 shutdown_connections 2024-09-05T01:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.553+0000 7f56dc02f740 1 -- 172.21.15.129:0/964924414 wait complete. 2024-09-05T01:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.553+0000 7f56dc02f740 1 -- start start 2024-09-05T01:24:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.553+0000 7f56dc02f740 1 Processor -- start 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff48540 0x5637eff51340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 0x5637effca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b36b0 con 0x5637eff54310 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b4b20 con 0x5637eff434b0 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56d41b3d80 con 0x5637eff48540 2024-09-05T01:24:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.554+0000 7f56dc02f740 10 client.0 _ll_register_callbacks cb 0x7f56c8097530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:24:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:33894/0 (socket says 172.21.15.129:33894) 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d2ffd640 1 -- 172.21.15.129:0/3638911206 learned_addr learned my addr 172.21.15.129:0/3638911206 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 0x5637effca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d27fc640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff48540 0x5637eff51340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 -- 172.21.15.129:0/3638911206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff48540 msgr2=0x5637eff51340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff48540 0x5637eff51340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 -- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 msgr2=0x5637eff50e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 -- 172.21.15.129:0/3638911206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637effbd800 con 0x5637eff54310 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56d37fe640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 0x5637effca030 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f56c403d450 tx=0x7f56c4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb721224ce7af63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.555+0000 7f56b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004d10 con 0x5637eff54310 2024-09-05T01:24:58.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5637efeb3a20 con 0x5637eff54310 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f56c4003b80 con 0x5637eff54310 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56c406f050 con 0x5637eff54310 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56dc02f740 10 client.16383 fetch_fsmap learned FSMap version 30 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5637effbeb70 con 0x5637eff54310 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5637efeb3a20 con 0x5637eff54310 2024-09-05T01:24:58.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.556+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f56c4068060 con 0x5637eff54310 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 20 client.16383 populate_metadata read hostname 'smithi129' 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637effbf4f0 0x5637effc1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 4 client.16383 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5637efeadd70 con 0x5637effbf4f0 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.557+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637effbf4f0 0x5637effc1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:24:58.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.558+0000 7f56d2ffd640 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637effbf4f0 0x5637effc1910 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96da52db5384ac3b server_cookie=943841aea1e1ab85 in_seq=0 out_seq=0 2024-09-05T01:24:58.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.558+0000 7f56b6ff5640 10 client.16383 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.604+0000 7f56b6ff5640 1 -- 172.21.15.129:0/3638911206 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5637efeadd70 con 0x5637effbf4f0 2024-09-05T01:24:58.605 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.605+0000 7f56b6ff5640 10 client.16383 handle_command_reply: tid=0 2024-09-05T01:24:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.605+0000 7f56dc02f740 1 client.16383 shutdown 2024-09-05T01:24:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 msgr2=0x5637effca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.606 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 0x5637effca030 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f56c403d450 tx=0x7f56c4009780 comp rx=0 tx=0).stop 2024-09-05T01:24:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 shutdown_connections 2024-09-05T01:24:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5637effbf4f0 0x5637effc1910 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.607 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5637eff54310 0x5637effca030 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5637eff48540 0x5637eff51340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 --2- 172.21.15.129:0/3638911206 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5637eff434b0 0x5637eff50e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 >> 172.21.15.129:0/3638911206 conn(0x5637eff3e1e0 msgr2=0x5637eff4cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.606+0000 7f56b6ff5640 0 client.16383 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.607+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 shutdown_connections 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.607+0000 7f56dc02f740 1 -- 172.21.15.129:0/3638911206 wait complete. 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.607+0000 7f56dc02f740 20 client.16383 trim_cache size 0 max 16384 2024-09-05T01:24:58.608 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:24:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:24:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:24:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 msgr2=0x7f56bc0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 0x7f56bc0a5180 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56c4005ab0 tx=0x7f56c4003ed0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 msgr2=0x7f56d41a2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f56c800d770 tx=0x7f56c800dc30 comp rx=0 tx=0).stop 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 shutdown_connections 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56bc0a2cc0 0x7f56bc0a5180 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d410df00 0x7f56d41a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.608+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d410d540 0x7f56d41a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.609+0000 7f56d3fff640 1 --2- 172.21.15.129:0/2200504906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d410c6b0 0x7f56d41a2560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:24:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.609+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 >> 172.21.15.129:0/2200504906 conn(0x7f56d406bda0 msgr2=0x7f56d41109b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:24:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.609+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 shutdown_connections 2024-09-05T01:24:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:24:58.609+0000 7f56d3fff640 1 -- 172.21.15.129:0/2200504906 wait complete. 2024-09-05T01:24:59.303 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:24:59 smithi129 ceph-mon[30979]: pgmap v641: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-05T01:24:59.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:24:59 smithi183 ceph-mon[34589]: pgmap v641: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-05T01:24:59.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:24:59 smithi194 ceph-mon[34941]: pgmap v641: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.1 MiB/s wr, 534 op/s 2024-09-05T01:25:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 475.0 s, 1020.6 tps, lat 97.387 ms stddev 32.687 2024-09-05T01:25:01.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:01 smithi183 ceph-mon[34589]: pgmap v642: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-09-05T01:25:01.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:01 smithi129 ceph-mon[30979]: pgmap v642: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-09-05T01:25:01.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:01 smithi194 ceph-mon[34941]: pgmap v642: 129 pgs: 129 active+clean; 1.3 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.0 MiB/s wr, 648 op/s 2024-09-05T01:25:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:03 smithi183 ceph-mon[34589]: pgmap v643: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.5 MiB/s wr, 817 op/s 2024-09-05T01:25:03.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:03 smithi129 ceph-mon[30979]: pgmap v643: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.5 MiB/s wr, 817 op/s 2024-09-05T01:25:03.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:03 smithi194 ceph-mon[34941]: pgmap v643: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.5 MiB/s wr, 817 op/s 2024-09-05T01:25:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 480.0 s, 988.6 tps, lat 101.381 ms stddev 36.221 2024-09-05T01:25:05.301 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:05 smithi194 ceph-mon[34941]: pgmap v644: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.2 MiB/s wr, 902 op/s 2024-09-05T01:25:05.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:05 smithi183 ceph-mon[34589]: pgmap v644: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.2 MiB/s wr, 902 op/s 2024-09-05T01:25:05.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:05 smithi129 ceph-mon[30979]: pgmap v644: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.2 MiB/s wr, 902 op/s 2024-09-05T01:25:07.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:07 smithi183 ceph-mon[34589]: pgmap v645: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 941 op/s 2024-09-05T01:25:07.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:07 smithi129 ceph-mon[30979]: pgmap v645: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 941 op/s 2024-09-05T01:25:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:07 smithi194 ceph-mon[34941]: pgmap v645: 129 pgs: 129 active+clean; 1.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 8.6 MiB/s wr, 941 op/s 2024-09-05T01:25:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:08 smithi183 ceph-mon[34589]: pgmap v646: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-05T01:25:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:08 smithi129 ceph-mon[30979]: pgmap v646: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-05T01:25:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:08 smithi194 ceph-mon[34941]: pgmap v646: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 9.8 MiB/s wr, 1.07k op/s 2024-09-05T01:25:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 485.0 s, 1007.8 tps, lat 99.388 ms stddev 43.942 2024-09-05T01:25:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:11 smithi183 ceph-mon[34589]: pgmap v647: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.5 MiB/s wr, 947 op/s 2024-09-05T01:25:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:11 smithi129 ceph-mon[30979]: pgmap v647: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.5 MiB/s wr, 947 op/s 2024-09-05T01:25:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:11 smithi194 ceph-mon[34941]: pgmap v647: 129 pgs: 129 active+clean; 1.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 8.5 MiB/s wr, 947 op/s 2024-09-05T01:25:11.763 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.763+0000 7f6656dfd640 1 -- 172.21.15.129:0/1665746365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650104f50 msgr2=0x7f66501053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.763+0000 7f6656dfd640 1 --2- 172.21.15.129:0/1665746365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650104f50 0x7f66501053d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6640009be0 tx=0x7f664002a6c0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.764+0000 7f6656dfd640 1 -- 172.21.15.129:0/1665746365 shutdown_connections 2024-09-05T01:25:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.765+0000 7f6656dfd640 1 --2- 172.21.15.129:0/1665746365 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650105910 0x7f665010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.765+0000 7f6656dfd640 1 --2- 172.21.15.129:0/1665746365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650104f50 0x7f66501053d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.765+0000 7f6656dfd640 1 --2- 172.21.15.129:0/1665746365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f6650104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.765+0000 7f6656dfd640 1 -- 172.21.15.129:0/1665746365 >> 172.21.15.129:0/1665746365 conn(0x7f66500ff4e0 msgr2=0x7f6650101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.766+0000 7f6656dfd640 1 -- 172.21.15.129:0/1665746365 shutdown_connections 2024-09-05T01:25:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.767+0000 7f6656dfd640 1 -- 172.21.15.129:0/1665746365 wait complete. 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 Processor -- start 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 -- start start 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f665019e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 0x7f665019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 0x7f66501a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f665010f8c0 con 0x7f6650103d60 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f665010f740 con 0x7f6650105910 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.768+0000 7f6656dfd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f665010fa40 con 0x7f6650104f50 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.769+0000 7f665539a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 0x7f66501a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.769+0000 7f665539a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 0x7f66501a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58848/0 (socket says 172.21.15.129:58848) 2024-09-05T01:25:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.769+0000 7f665539a640 1 -- 172.21.15.129:0/3241351426 learned_addr learned my addr 172.21.15.129:0/3241351426 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.769+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f665019e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.769+0000 7f664ffff640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 0x7f665019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 -- 172.21.15.129:0/3241351426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 msgr2=0x7f665019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 0x7f665019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 -- 172.21.15.129:0/3241351426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 msgr2=0x7f66501a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 0x7f66501a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 -- 172.21.15.129:0/3241351426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66501950d0 con 0x7f6650103d60 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f664ffff640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 0x7f665019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.770+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f665019e330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9c0 tx=0x7f663800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.771+0000 7f664dffb640 1 -- 172.21.15.129:0/3241351426 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800ccc0 con 0x7f6650103d60 2024-09-05T01:25:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.771+0000 7f664dffb640 1 -- 172.21.15.129:0/3241351426 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66380035f0 con 0x7f6650103d60 2024-09-05T01:25:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.771+0000 7f664dffb640 1 -- 172.21.15.129:0/3241351426 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663800ce60 con 0x7f6650103d60 2024-09-05T01:25:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.771+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66501953c0 con 0x7f6650103d60 2024-09-05T01:25:11.773 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.771+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66501958c0 con 0x7f6650103d60 2024-09-05T01:25:11.775 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.774+0000 7f6659c7e740 1 Processor -- start 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 -- start start 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b82e0 0x5568243bcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243bf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55682431b2f0 con 0x5568243bd370 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568242c3270 con 0x5568243b79a0 2024-09-05T01:25:11.776 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f6659c7e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5568243172b0 con 0x5568243b82e0 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f665539a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f665539a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58852/0 (socket says 172.21.15.129:58852) 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.775+0000 7f665539a640 1 -- 172.21.15.129:0/835080873 learned_addr learned my addr 172.21.15.129:0/835080873 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f664ffff640 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243bf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 -- 172.21.15.129:0/835080873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b82e0 msgr2=0x5568243bcda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b82e0 0x5568243bcda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 -- 172.21.15.129:0/835080873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 msgr2=0x5568243bf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.777 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243bf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 -- 172.21.15.129:0/835080873 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568242c2340 con 0x5568243b79a0 2024-09-05T01:25:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f664ffff640 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:25:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.776+0000 7f665539a640 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6648009d60 tx=0x7f6648011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63d7bcf5c20fee1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.778+0000 7f6646ffd640 1 -- 172.21.15.129:0/835080873 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801e070 con 0x5568243b79a0 2024-09-05T01:25:11.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.778+0000 7f6646ffd640 1 -- 172.21.15.129:0/835080873 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6648019ae0 con 0x5568243b79a0 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.778+0000 7f664dffb640 1 -- 172.21.15.129:0/3241351426 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f66380037a0 con 0x7f6650103d60 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.778+0000 7f6646ffd640 1 -- 172.21.15.129:0/835080873 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664800ea70 con 0x5568243b79a0 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 -- 172.21.15.129:0/835080873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 msgr2=0x5568243b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6648009d60 tx=0x7f6648011810 comp rx=0 tx=0).stop 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f664dffb640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 0x7f663c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 -- 172.21.15.129:0/835080873 shutdown_connections 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243bf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f664dffb640 1 -- 172.21.15.129:0/3241351426 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f6638099e50 con 0x7f6650103d60 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b82e0 0x5568243bcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 --2- 172.21.15.129:0/835080873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243b79a0 0x5568243b7da0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.779+0000 7f6659c7e740 1 -- 172.21.15.129:0/835080873 >> 172.21.15.129:0/835080873 conn(0x5568243b3010 msgr2=0x5568243b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.780+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 0x7f663c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.780+0000 7f6659c7e740 1 -- 172.21.15.129:0/835080873 shutdown_connections 2024-09-05T01:25:11.782 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.780+0000 7f6659c7e740 1 -- 172.21.15.129:0/835080873 wait complete. 2024-09-05T01:25:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.781+0000 7f6654b99640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 0x7f663c0a51c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6638099200 tx=0x7f66380a1280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.781+0000 7f6659c7e740 1 -- start start 2024-09-05T01:25:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.781+0000 7f6659c7e740 1 Processor -- start 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48252/0 (socket says 172.21.15.129:48252) 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.782+0000 7f665539a640 1 -- 172.21.15.129:0/744037170 learned_addr learned my addr 172.21.15.129:0/744037170 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.783+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5568243c03e0 con 0x5568243bd370 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.783+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5568242a4290 con 0x5568243c72c0 2024-09-05T01:25:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.783+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5568243c0560 con 0x5568243b79a0 2024-09-05T01:25:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.783+0000 7f6659c7e740 10 client.0 _ll_register_callbacks cb 0x5568242d5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.783+0000 7f665539a640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 -- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 msgr2=0x5568243c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 -- 172.21.15.129:0/744037170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 msgr2=0x5568243c2290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f664ffff640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682402e210 con 0x5568243b79a0 2024-09-05T01:25:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f665539a640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6648019aa0 tx=0x7f66480099a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db235bd3f039788d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f662ffff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.784+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664802b060 con 0x5568243b79a0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.785+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f664800e990 con 0x5568243b79a0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.785+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6648019ae0 con 0x5568243b79a0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.785+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6648032050 con 0x5568243b79a0 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.785+0000 7f6659c7e740 10 client.25849 fetch_fsmap learned FSMap version 30 2024-09-05T01:25:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.785+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5568240fa620 con 0x5568243b79a0 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6648019ae0 con 0x5568243b79a0 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6648023380 con 0x5568243b79a0 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 10 client.25849 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 10 client.25849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 20 client.25849 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5568243cd9c0 0x5568243cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 4 client.25849 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:25:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556824328a20 con 0x5568243cd9c0 2024-09-05T01:25:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.786+0000 7f664ffff640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5568243cd9c0 0x5568243cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.787+0000 7f664ffff640 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5568243cd9c0 0x5568243cfdb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db330a2f7afab190 server_cookie=961029b8a607de4d in_seq=0 out_seq=0 2024-09-05T01:25:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.789+0000 7f662ffff640 10 client.25849 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.789+0000 7f662ffff640 1 -- 172.21.15.129:0/744037170 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556824328a20 con 0x5568243cd9c0 2024-09-05T01:25:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.789+0000 7f662ffff640 10 client.25849 handle_command_reply: tid=0 2024-09-05T01:25:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.790+0000 7f6659c7e740 1 client.25849 shutdown 2024-09-05T01:25:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 msgr2=0x5568243c1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6648019aa0 tx=0x7f66480099a0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 shutdown_connections 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5568243cd9c0 0x5568243cfdb0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 --2- 172.21.15.129:0/744037170 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243c1d50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 >> 172.21.15.129:0/744037170 conn(0x5568243b3010 msgr2=0x5568243b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.791+0000 7f662ffff640 0 client.25849 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.792+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 shutdown_connections 2024-09-05T01:25:11.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.792+0000 7f6659c7e740 1 -- 172.21.15.129:0/744037170 wait complete. 2024-09-05T01:25:11.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.793+0000 7f6659c7e740 20 client.25849 trim_cache size 0 max 16384 2024-09-05T01:25:11.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.825+0000 7f6659c7e740 1 Processor -- start 2024-09-05T01:25:11.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.825+0000 7f6659c7e740 1 -- start start 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.825+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.825+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243bd370 0x5568243b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48262/0 (socket says 172.21.15.129:48262) 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f665539a640 1 -- 172.21.15.129:0/539602882 learned_addr learned my addr 172.21.15.129:0/539602882 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f664ffff640 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243bd370 0x5568243b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f6659c7e740 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243c72c0 0x5568243b4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66501bc0b0 con 0x5568243c72c0 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b9980 con 0x5568243bd370 2024-09-05T01:25:11.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.826+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b7800 con 0x5568243b79a0 2024-09-05T01:25:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243c72c0 0x5568243b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 -- 172.21.15.129:0/539602882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243bd370 msgr2=0x5568243b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243bd370 0x5568243b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 -- 172.21.15.129:0/539602882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243c72c0 msgr2=0x5568243b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243c72c0 0x5568243b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 -- 172.21.15.129:0/539602882 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568243b3ca0 con 0x5568243b79a0 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f665539a640 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6648023780 tx=0x7f6648011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf30a25bc727679c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.827+0000 7f66467fc640 1 -- 172.21.15.129:0/539602882 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664802b040 con 0x5568243b79a0 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.828+0000 7f66467fc640 1 -- 172.21.15.129:0/539602882 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6648005630 con 0x5568243b79a0 2024-09-05T01:25:11.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.828+0000 7f66467fc640 1 -- 172.21.15.129:0/539602882 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664801e040 con 0x5568243b79a0 2024-09-05T01:25:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 msgr2=0x5568243b7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6648023780 tx=0x7f6648011810 comp rx=0 tx=0).stop 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 shutdown_connections 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243c72c0 0x5568243b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243bd370 0x5568243b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 --2- 172.21.15.129:0/539602882 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243b7d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.829+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 >> 172.21.15.129:0/539602882 conn(0x5568243b3010 msgr2=0x5568243c1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.830+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 shutdown_connections 2024-09-05T01:25:11.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.830+0000 7f6659c7e740 1 -- 172.21.15.129:0/539602882 wait complete. 2024-09-05T01:25:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.830+0000 7f6659c7e740 1 -- start start 2024-09-05T01:25:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.830+0000 7f6659c7e740 1 Processor -- start 2024-09-05T01:25:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f6659c7e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243df930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f665539a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48276/0 (socket says 172.21.15.129:48276) 2024-09-05T01:25:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f665539a640 1 -- 172.21.15.129:0/2002537061 learned_addr learned my addr 172.21.15.129:0/2002537061 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:11.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243e0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.832+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b7e40 con 0x5568243bd370 2024-09-05T01:25:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.833+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b92b0 con 0x5568243c72c0 2024-09-05T01:25:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.833+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f66501b8510 con 0x5568243b79a0 2024-09-05T01:25:11.834 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.833+0000 7f6659c7e740 10 client.0 _ll_register_callbacks cb 0x7f664801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:11.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f664ffff640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243e0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 -- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 msgr2=0x5568243e0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243e0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 -- 172.21.15.129:0/2002537061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 msgr2=0x5568243df930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243df930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568243e0f50 con 0x5568243b79a0 2024-09-05T01:25:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.835+0000 7f665539a640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6648005ab0 tx=0x7f6648023970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b62ef862b2b7e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.836+0000 7f662ffff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:25:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664803b040 con 0x5568243b79a0 2024-09-05T01:25:11.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9266+0+0 (secure 0 0 0) 0x7f6648009970 con 0x5568243b79a0 2024-09-05T01:25:11.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556824322d70 con 0x5568243b79a0 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f664804c050 con 0x5568243b79a0 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f6659c7e740 10 client.25855 fetch_fsmap learned FSMap version 30 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.837+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556824438510 con 0x5568243b79a0 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6648023590 con 0x5568243b79a0 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 30) ==== 8354+0+0 (secure 0 0 0) 0x7f6648041070 con 0x5568243b79a0 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 10 client.25855 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:11.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 20 client.25855 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556824438d60 0x55682443b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 4 client.25855 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:25:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556824322d70 con 0x556824438d60 2024-09-05T01:25:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.838+0000 7f664ffff640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556824438d60 0x55682443b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:11.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.839+0000 7f662ffff640 10 client.25855 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:11.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.840+0000 7f664ffff640 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556824438d60 0x55682443b180 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1ca9546eef5f39e server_cookie=9685256524a1e422 in_seq=0 out_seq=0 2024-09-05T01:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.865+0000 7f662ffff640 1 -- 172.21.15.129:0/2002537061 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556824322d70 con 0x556824438d60 2024-09-05T01:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.865+0000 7f662ffff640 10 client.25855 handle_command_reply: tid=0 2024-09-05T01:25:11.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.865+0000 7f6659c7e740 1 client.25855 shutdown 2024-09-05T01:25:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.866+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 msgr2=0x5568243df3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.866+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6648005ab0 tx=0x7f6648023970 comp rx=0 tx=0).stop 2024-09-05T01:25:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.867+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 shutdown_connections 2024-09-05T01:25:11.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.867+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556824438d60 0x55682443b180 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.867+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5568243c72c0 0x5568243e0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.868+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5568243bd370 0x5568243df930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.868+0000 7f6659c7e740 1 --2- 172.21.15.129:0/2002537061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5568243b79a0 0x5568243df3f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.868+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 >> 172.21.15.129:0/2002537061 conn(0x5568243b3010 msgr2=0x5568243c1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.868+0000 7f662ffff640 0 client.25855 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.869+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 shutdown_connections 2024-09-05T01:25:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.869+0000 7f6659c7e740 1 -- 172.21.15.129:0/2002537061 wait complete. 2024-09-05T01:25:11.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.869+0000 7f6659c7e740 20 client.25855 trim_cache size 0 max 16384 2024-09-05T01:25:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:25:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:25:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:25:11.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:25:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 msgr2=0x7f663c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 0x7f663c0a51c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6638099200 tx=0x7f66380a1280 comp rx=0 tx=0).stop 2024-09-05T01:25:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 msgr2=0x7f665019e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f665019e330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f663800e9c0 tx=0x7f663800ee80 comp rx=0 tx=0).stop 2024-09-05T01:25:11.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 shutdown_connections 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f663c0a2d00 0x7f663c0a51c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6650105910 0x7f66501a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6650104f50 0x7f665019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 --2- 172.21.15.129:0/3241351426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6650103d60 0x7f665019e330 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.872+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 >> 172.21.15.129:0/3241351426 conn(0x7f66500ff4e0 msgr2=0x7f6650078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.873+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 shutdown_connections 2024-09-05T01:25:11.874 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:11.873+0000 7f6656dfd640 1 -- 172.21.15.129:0/3241351426 wait complete. 2024-09-05T01:25:12.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:12 smithi194 ceph-mon[34941]: daemon mds.c restarted 2024-09-05T01:25:12.251 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:25:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setuser ceph since I am not root 2024-09-05T01:25:12.251 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:25:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setgroup ceph since I am not root 2024-09-05T01:25:12.251 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:25:11 smithi194 ceph-mds[65377]: -- 172.21.15.194:0/2558934692 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5604bc077c20 con 0x5604bc055000 2024-09-05T01:25:12.251 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:25:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: starting mds.c at 2024-09-05T01:25:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:12 smithi183 ceph-mon[34589]: daemon mds.c restarted 2024-09-05T01:25:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:12 smithi129 ceph-mon[30979]: daemon mds.c restarted 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: pgmap v648: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 964 op/s 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6836/4061055180,v1:172.21.15.194:6837/4061055180] up:boot 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:25:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:13 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:25:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: pgmap v648: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 964 op/s 2024-09-05T01:25:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T01:25:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6836/4061055180,v1:172.21.15.194:6837/4061055180] up:boot 2024-09-05T01:25:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:25:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:25:13.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:13 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:25:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: pgmap v648: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 8.7 MiB/s wr, 964 op/s 2024-09-05T01:25:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T01:25:13.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6836/4061055180,v1:172.21.15.194:6837/4061055180] up:boot 2024-09-05T01:25:13.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:25:13.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:25:13.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:13 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:25:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:14 smithi129 ceph-mon[30979]: pgmap v650: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 937 op/s 2024-09-05T01:25:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:14 smithi194 ceph-mon[34941]: pgmap v650: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 937 op/s 2024-09-05T01:25:14.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:14 smithi183 ceph-mon[34589]: pgmap v650: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.5 MiB/s wr, 937 op/s 2024-09-05T01:25:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 490.0 s, 887.2 tps, lat 112.947 ms stddev 36.180 2024-09-05T01:25:15.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:15 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:25:15.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:15 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:25:16.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:15 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:25:16.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:16 smithi129 ceph-mon[30979]: pgmap v651: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 955 op/s 2024-09-05T01:25:16.911 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:16 smithi183 ceph-mon[34589]: pgmap v651: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 955 op/s 2024-09-05T01:25:17.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:16 smithi194 ceph-mon[34941]: pgmap v651: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.8 MiB/s wr, 955 op/s 2024-09-05T01:25:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:18 smithi183 ceph-mon[34589]: pgmap v652: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 905 op/s 2024-09-05T01:25:18.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:18 smithi129 ceph-mon[30979]: pgmap v652: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 905 op/s 2024-09-05T01:25:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:18.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:18 smithi194 ceph-mon[34941]: pgmap v652: 129 pgs: 129 active+clean; 1.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.1 MiB/s wr, 905 op/s 2024-09-05T01:25:19.394 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:25:19.394 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:25:19.394 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:25:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:25:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:25:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:25:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:25:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 495.0 s, 936.4 tps, lat 106.729 ms stddev 40.237 2024-09-05T01:25:20.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:20 smithi183 ceph-mon[34589]: pgmap v653: 129 pgs: 129 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 859 op/s 2024-09-05T01:25:20.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:20 smithi129 ceph-mon[30979]: pgmap v653: 129 pgs: 129 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 859 op/s 2024-09-05T01:25:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:20 smithi194 ceph-mon[34941]: pgmap v653: 129 pgs: 129 active+clean; 1.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.5 MiB/s wr, 859 op/s 2024-09-05T01:25:23.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:23 smithi183 ceph-mon[34589]: pgmap v654: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 906 op/s 2024-09-05T01:25:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:23 smithi129 ceph-mon[30979]: pgmap v654: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 906 op/s 2024-09-05T01:25:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:23 smithi194 ceph-mon[34941]: pgmap v654: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 8.1 MiB/s wr, 906 op/s 2024-09-05T01:25:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.026+0000 7f3ebc56d640 1 -- 172.21.15.129:0/3129693480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109d40 msgr2=0x7f3eb41105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.026+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/3129693480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109d40 0x7f3eb41105d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009860 tx=0x7f3ea402a360 comp rx=0 tx=0).stop 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 -- 172.21.15.129:0/3129693480 shutdown_connections 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/3129693480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109d40 0x7f3eb41105d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/3129693480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109380 0x7f3eb4109800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/3129693480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb4108470 0x7f3eb4108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 -- 172.21.15.129:0/3129693480 >> 172.21.15.129:0/3129693480 conn(0x7f3eb4077470 msgr2=0x7f3eb40798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 -- 172.21.15.129:0/3129693480 shutdown_connections 2024-09-05T01:25:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.028+0000 7f3ebc56d640 1 -- 172.21.15.129:0/3129693480 wait complete. 2024-09-05T01:25:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 Processor -- start 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 -- start start 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb4108470 0x7f3eb419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 0x7f3eb41a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4113d20 con 0x7f3eb4109d40 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4113ba0 con 0x7f3eb4109380 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3ebc56d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4113ea0 con 0x7f3eb4108470 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3eb9b08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3eb9b08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:39810/0 (socket says 172.21.15.129:39810) 2024-09-05T01:25:25.032 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.030+0000 7f3eb9b08640 1 -- 172.21.15.129:0/557982646 learned_addr learned my addr 172.21.15.129:0/557982646 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 -- 172.21.15.129:0/557982646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb4108470 msgr2=0x7f3eb419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb4108470 0x7f3eb419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 -- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 msgr2=0x7f3eb41a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 0x7f3eb41a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 0x7f3eb41a58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 -- 172.21.15.129:0/557982646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb407ab70 con 0x7f3eb4109380 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 0x7f3eb41a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eb9b08640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e9f0 tx=0x7f3eb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.033 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.031+0000 7f3eab7fe640 1 -- 172.21.15.129:0/557982646 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000ccc0 con 0x7f3eb4109380 2024-09-05T01:25:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.032+0000 7f3eab7fe640 1 -- 172.21.15.129:0/557982646 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3eb00035f0 con 0x7f3eb4109380 2024-09-05T01:25:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.032+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb407ae60 con 0x7f3eb4109380 2024-09-05T01:25:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.032+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb407b400 con 0x7f3eb4109380 2024-09-05T01:25:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.032+0000 7f3eab7fe640 1 -- 172.21.15.129:0/557982646 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb000ce60 con 0x7f3eb4109380 2024-09-05T01:25:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.036+0000 7f3eab7fe640 1 -- 172.21.15.129:0/557982646 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3eb00037a0 con 0x7f3eb4109380 2024-09-05T01:25:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.038+0000 7f3ebf3ea740 1 Processor -- start 2024-09-05T01:25:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3ebf3ea740 1 -- start start 2024-09-05T01:25:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0efda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3eab7fe640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 0x7f3ea00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0efda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.039+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0efda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:39816/0 (socket says 172.21.15.129:39816) 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0f7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55979e0532f0 con 0x55979e0f02e0 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55979dffb270 con 0x55979e0ef9a0 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55979e04f2b0 con 0x55979e0f5370 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eb9b08640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 0x7f3ea00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35908/0 (socket says 172.21.15.129:35908) 2024-09-05T01:25:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 -- 172.21.15.129:0/1021840309 learned_addr learned my addr 172.21.15.129:0/1021840309 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eab7fe640 1 -- 172.21.15.129:0/557982646 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f3eb00cf780 con 0x7f3eb4109380 2024-09-05T01:25:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eb9b08640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 0x7f3ea00a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00978b0 tx=0x7f3eb0097820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 -- 172.21.15.129:0/1021840309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 msgr2=0x55979e0f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 -- 172.21.15.129:0/1021840309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 msgr2=0x55979e0efda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0efda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.040+0000 7f3eba309640 1 -- 172.21.15.129:0/1021840309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979dffa340 con 0x55979e0f02e0 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.041+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.041+0000 7f3eba309640 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009da0 tx=0x7f3eac00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af128637960bf350 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.041+0000 7f3ea8bfe640 1 -- 172.21.15.129:0/1021840309 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac00bd40 con 0x55979e0f02e0 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.041+0000 7f3ea8bfe640 1 -- 172.21.15.129:0/1021840309 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3eac017070 con 0x55979e0f02e0 2024-09-05T01:25:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.041+0000 7f3ea8bfe640 1 -- 172.21.15.129:0/1021840309 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011700 con 0x55979e0f02e0 2024-09-05T01:25:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/1021840309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 msgr2=0x55979e0f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009da0 tx=0x7f3eac00b420 comp rx=0 tx=0).stop 2024-09-05T01:25:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/1021840309 shutdown_connections 2024-09-05T01:25:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0f4da0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/1021840309 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0efda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/1021840309 >> 172.21.15.129:0/1021840309 conn(0x55979e0eb010 msgr2=0x55979e0eb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.042+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/1021840309 shutdown_connections 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.043+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/1021840309 wait complete. 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.043+0000 7f3ebf3ea740 1 -- start start 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.043+0000 7f3ebf3ea740 1 Processor -- start 2024-09-05T01:25:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.044+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.044+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.044+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:39820/0 (socket says 172.21.15.129:39820) 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.044+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.044+0000 7f3ebab0a640 1 -- 172.21.15.129:0/768292537 learned_addr learned my addr 172.21.15.129:0/768292537 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e1011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55979e0f8340 con 0x55979e0f02e0 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3eba309640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55979dfdc290 con 0x55979e0ef9a0 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55979e0f84c0 con 0x55979e0f5370 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.045+0000 7f3ebf3ea740 10 client.0 _ll_register_callbacks cb 0x55979e00d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e1011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 -- 172.21.15.129:0/768292537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 msgr2=0x55979e1011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e1011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 -- 172.21.15.129:0/768292537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 msgr2=0x55979e0fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979dd66210 con 0x55979e0ef9a0 2024-09-05T01:25:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009830 tx=0x7f3ea4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1510e596bc5ef9e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.046+0000 7f3e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.049+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4046020 con 0x55979e0ef9a0 2024-09-05T01:25:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.049+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f3ea4004240 con 0x55979e0ef9a0 2024-09-05T01:25:25.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.049+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eac017070 con 0x55979e0ef9a0 2024-09-05T01:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ea4051050 con 0x55979e0ef9a0 2024-09-05T01:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3ebf3ea740 10 client.25955 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55979de32620 con 0x55979e0ef9a0 2024-09-05T01:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ea4004440 con 0x55979e0ef9a0 2024-09-05T01:25:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ea4032670 con 0x55979e0ef9a0 2024-09-05T01:25:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3ebf3ea740 10 client.25955 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3ebf3ea740 10 client.25955 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.051+0000 7f3ebf3ea740 20 client.25955 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.052+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.052+0000 7f3ebf3ea740 4 client.25955 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:25:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.052+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3eac017070 con 0x55979e105830 2024-09-05T01:25:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.052+0000 7f3eba309640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.052+0000 7f3e9dffb640 10 client.25955 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:25.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.053+0000 7f3eba309640 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e107c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a81f6f3aaf09b60 server_cookie=588c6531f05f37e1 in_seq=0 out_seq=0 2024-09-05T01:25:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.053+0000 7f3e9dffb640 1 -- 172.21.15.129:0/768292537 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f3eac017070 con 0x55979e105830 2024-09-05T01:25:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.053+0000 7f3e9dffb640 10 client.25955 handle_command_reply: tid=0 2024-09-05T01:25:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.053+0000 7f3ebf3ea740 1 client.25955 shutdown 2024-09-05T01:25:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 msgr2=0x55979e0f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009830 tx=0x7f3ea4005ab0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 shutdown_connections 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e107c40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e1011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3e9dffb640 0 client.25955 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e0fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/768292537 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0f9c30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 >> 172.21.15.129:0/768292537 conn(0x55979e0eb010 msgr2=0x55979e0ec820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 shutdown_connections 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/768292537 wait complete. 2024-09-05T01:25:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.054+0000 7f3ebf3ea740 20 client.25955 trim_cache size 0 max 16384 2024-09-05T01:25:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.088+0000 7f3ebf3ea740 1 Processor -- start 2024-09-05T01:25:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.088+0000 7f3ebf3ea740 1 -- start start 2024-09-05T01:25:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.088+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0ebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.089 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.088+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.088+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0ec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3eba309640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41bc070 con 0x55979e0f02e0 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b9940 con 0x55979e0ef9a0 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3ebf3ea740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b77b0 con 0x55979e0f5370 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3eba309640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35928/0 (socket says 172.21.15.129:35928) 2024-09-05T01:25:25.090 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.089+0000 7f3eba309640 1 -- 172.21.15.129:0/3062939945 learned_addr learned my addr 172.21.15.129:0/3062939945 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.090+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0ebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 -- 172.21.15.129:0/3062939945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 msgr2=0x55979e0ec400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0ec400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 -- 172.21.15.129:0/3062939945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 msgr2=0x55979e0ebec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0ebec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 -- 172.21.15.129:0/3062939945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979e0f8640 con 0x55979e0f02e0 2024-09-05T01:25:25.092 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.091+0000 7f3eba309640 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b920 tx=0x7f3eac012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5dfd6a49db815c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.092+0000 7f3e9ffff640 1 -- 172.21.15.129:0/3062939945 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac012c80 con 0x55979e0f02e0 2024-09-05T01:25:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.092+0000 7f3e9ffff640 1 -- 172.21.15.129:0/3062939945 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3eac017040 con 0x55979e0f02e0 2024-09-05T01:25:25.093 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.092+0000 7f3e9ffff640 1 -- 172.21.15.129:0/3062939945 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac027920 con 0x55979e0f02e0 2024-09-05T01:25:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.093+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3062939945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 msgr2=0x55979e101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.093+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b920 tx=0x7f3eac012c10 comp rx=0 tx=0).stop 2024-09-05T01:25:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3062939945 shutdown_connections 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e0ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0f02e0 0x55979e101170 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3062939945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0ef9a0 0x55979e0ebec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3062939945 >> 172.21.15.129:0/3062939945 conn(0x55979e0eb010 msgr2=0x55979e0f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3062939945 shutdown_connections 2024-09-05T01:25:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3062939945 wait complete. 2024-09-05T01:25:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.094+0000 7f3ebf3ea740 1 -- start start 2024-09-05T01:25:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.095+0000 7f3ebf3ea740 1 Processor -- start 2024-09-05T01:25:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.095+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 0x55979e0feb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.095+0000 7f3ebf3ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 0x55979e0ff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 0x55979e0feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebab0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 0x55979e0feb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35938/0 (socket says 172.21.15.129:35938) 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebab0a640 1 -- 172.21.15.129:0/3113226666 learned_addr learned my addr 172.21.15.129:0/3113226666 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3eba309640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 0x55979e0ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e100080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b7e00 con 0x55979e0ef9a0 2024-09-05T01:25:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b9270 con 0x55979e0f02e0 2024-09-05T01:25:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb41b84d0 con 0x55979e0f5370 2024-09-05T01:25:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebf3ea740 10 client.0 _ll_register_callbacks cb 0x7f3eb0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:25.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.096+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 -- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 msgr2=0x55979e100080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e100080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 -- 172.21.15.129:0/3113226666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 msgr2=0x55979e0feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 0x55979e0feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55979e100690 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3eba309640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 0x55979e0ff070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55979e0fabf0 tx=0x7f3eac009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b37b80f614d8dbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3e9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eac012640 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.097+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f3eac00f040 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.098+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eac017040 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.098+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3eac0127e0 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.098+0000 7f3ebf3ea740 10 client.25961 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.098+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55979e1706d0 con 0x55979e0f02e0 2024-09-05T01:25:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.099+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:25:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.099+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3eac012980 con 0x55979e0f02e0 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.099+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3eac011070 con 0x55979e0f02e0 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 10 client.25961 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 10 client.25961 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 20 client.25961 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e172de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 4 client.25961 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f3eac017040 con 0x55979e105830 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.100+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e172de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.101+0000 7f3ebab0a640 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e172de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e034611b9d1a00 server_cookie=7e1515433bfb7996 in_seq=0 out_seq=0 2024-09-05T01:25:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.101+0000 7f3e9dffb640 10 client.25961 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.120+0000 7f3e9dffb640 1 -- 172.21.15.129:0/3113226666 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f3eac017040 con 0x55979e105830 2024-09-05T01:25:25.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.120+0000 7f3e9dffb640 10 client.25961 handle_command_reply: tid=0 2024-09-05T01:25:25.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.122+0000 7f3ebf3ea740 1 client.25961 shutdown 2024-09-05T01:25:25.130 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.126+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 msgr2=0x55979e0ff070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.126+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 0x55979e0ff070 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55979e0fabf0 tx=0x7f3eac009690 comp rx=0 tx=0).stop 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 shutdown_connections 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55979e105830 0x55979e172de0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3e9dffb640 0 client.25961 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55979e0f5370 0x55979e100080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55979e0f02e0 0x55979e0ff070 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 --2- 172.21.15.129:0/3113226666 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55979e0ef9a0 0x55979e0feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.127+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 >> 172.21.15.129:0/3113226666 conn(0x55979e0eb010 msgr2=0x55979e0e99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.128+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 shutdown_connections 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.128+0000 7f3ebf3ea740 1 -- 172.21.15.129:0/3113226666 wait complete. 2024-09-05T01:25:25.131 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.129+0000 7f3ebf3ea740 20 client.25961 trim_cache size 0 max 16384 2024-09-05T01:25:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:25:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:25:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:25:25.132 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:25:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.133+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 msgr2=0x7f3ea00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.133+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 0x7f3ea00a5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3eb00978b0 tx=0x7f3eb0097820 comp rx=0 tx=0).stop 2024-09-05T01:25:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.134+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 msgr2=0x7f3eb419e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.134+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3eb000e9f0 tx=0x7f3eb000eeb0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 shutdown_connections 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f3ea00a2cb0 0x7f3ea00a5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3eb4109d40 0x7f3eb41a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3eb4109380 0x7f3eb419e860 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 --2- 172.21.15.129:0/557982646 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3eb4108470 0x7f3eb419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.135+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 >> 172.21.15.129:0/557982646 conn(0x7f3eb4077470 msgr2=0x7f3eb410c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.136+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 shutdown_connections 2024-09-05T01:25:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:25.136+0000 7f3ebc56d640 1 -- 172.21.15.129:0/557982646 wait complete. 2024-09-05T01:25:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 500.0 s, 968.2 tps, lat 103.197 ms stddev 33.497 2024-09-05T01:25:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:25 smithi183 ceph-mon[34589]: pgmap v655: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 895 op/s 2024-09-05T01:25:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:25 smithi129 ceph-mon[30979]: pgmap v655: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 895 op/s 2024-09-05T01:25:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:25 smithi194 ceph-mon[34941]: pgmap v655: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 895 op/s 2024-09-05T01:25:27.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:27 smithi183 ceph-mon[34589]: pgmap v656: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-05T01:25:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:27 smithi129 ceph-mon[30979]: pgmap v656: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-05T01:25:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:27 smithi194 ceph-mon[34941]: pgmap v656: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.7 MiB/s wr, 862 op/s 2024-09-05T01:25:29.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:29 smithi183 ceph-mon[34589]: pgmap v657: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.6 MiB/s wr, 977 op/s 2024-09-05T01:25:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:29 smithi129 ceph-mon[30979]: pgmap v657: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.6 MiB/s wr, 977 op/s 2024-09-05T01:25:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:29 smithi194 ceph-mon[34941]: pgmap v657: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.6 MiB/s wr, 977 op/s 2024-09-05T01:25:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 505.0 s, 931.8 tps, lat 107.254 ms stddev 32.711 2024-09-05T01:25:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:30 smithi183 ceph-mon[34589]: pgmap v658: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 881 op/s 2024-09-05T01:25:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:30 smithi129 ceph-mon[30979]: pgmap v658: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 881 op/s 2024-09-05T01:25:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:30 smithi194 ceph-mon[34941]: pgmap v658: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.7 MiB/s wr, 881 op/s 2024-09-05T01:25:33.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:33 smithi183 ceph-mon[34589]: pgmap v659: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 947 op/s 2024-09-05T01:25:33.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:33 smithi129 ceph-mon[30979]: pgmap v659: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 947 op/s 2024-09-05T01:25:33.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:33 smithi194 ceph-mon[34941]: pgmap v659: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.4 MiB/s wr, 947 op/s 2024-09-05T01:25:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:34 smithi129 ceph-mon[30979]: pgmap v660: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 911 op/s 2024-09-05T01:25:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:34 smithi194 ceph-mon[34941]: pgmap v660: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 911 op/s 2024-09-05T01:25:34.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:34 smithi183 ceph-mon[34589]: pgmap v660: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.9 MiB/s wr, 911 op/s 2024-09-05T01:25:35.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 510.0 s, 990.5 tps, lat 101.127 ms stddev 36.048 2024-09-05T01:25:37.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:37 smithi183 ceph-mon[34589]: pgmap v661: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 892 op/s 2024-09-05T01:25:37.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:37 smithi129 ceph-mon[30979]: pgmap v661: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 892 op/s 2024-09-05T01:25:37.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:37 smithi194 ceph-mon[34941]: pgmap v661: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 892 op/s 2024-09-05T01:25:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 -- 172.21.15.129:0/4276197608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 msgr2=0x7fa198103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 --2- 172.21.15.129:0/4276197608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa198103660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa190009ca0 tx=0x7fa19002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 -- 172.21.15.129:0/4276197608 shutdown_connections 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 --2- 172.21.15.129:0/4276197608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 --2- 172.21.15.129:0/4276197608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 0x7fa198106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 --2- 172.21.15.129:0/4276197608 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa198103660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.289+0000 7fa19feaf640 1 -- 172.21.15.129:0/4276197608 >> 172.21.15.129:0/4276197608 conn(0x7fa1980fb500 msgr2=0x7fa1980fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.290+0000 7fa19feaf640 1 -- 172.21.15.129:0/4276197608 shutdown_connections 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.290+0000 7fa19feaf640 1 -- 172.21.15.129:0/4276197608 wait complete. 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 Processor -- start 2024-09-05T01:25:38.291 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 -- start start 2024-09-05T01:25:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa19819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 0x7fa1981a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810f950 con 0x7fa198103c30 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810f7d0 con 0x7fa198100000 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19feaf640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa19810fad0 con 0x7fa198101270 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35160/0 (socket says 172.21.15.129:35160) 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19dc4b640 1 -- 172.21.15.129:0/1331427724 learned_addr learned my addr 172.21.15.129:0/1331427724 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.291+0000 7fa19d44a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa19819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19e44c640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 0x7fa1981a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 -- 172.21.15.129:0/1331427724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 msgr2=0x7fa19819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa19819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 -- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 msgr2=0x7fa1981a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 0x7fa1981a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 -- 172.21.15.129:0/1331427724 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1981a1c00 con 0x7fa198100000 2024-09-05T01:25:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa19002aeb0 tx=0x7fa190003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.292+0000 7fa18effd640 1 -- 172.21.15.129:0/1331427724 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190032ec0 con 0x7fa198100000 2024-09-05T01:25:38.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.293+0000 7fa18effd640 1 -- 172.21.15.129:0/1331427724 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1900044f0 con 0x7fa198100000 2024-09-05T01:25:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.293+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1981a1e90 con 0x7fa198100000 2024-09-05T01:25:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.293+0000 7fa18effd640 1 -- 172.21.15.129:0/1331427724 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190030b40 con 0x7fa198100000 2024-09-05T01:25:38.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.293+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1981a22a0 con 0x7fa198100000 2024-09-05T01:25:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.296+0000 7fa1a2c3f740 1 Processor -- start 2024-09-05T01:25:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.297+0000 7fa18effd640 1 -- 172.21.15.129:0/1331427724 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa19004e440 con 0x7fa198100000 2024-09-05T01:25:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.298+0000 7fa18effd640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 0x7fa1840a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa18effd640 1 -- 172.21.15.129:0/1331427724 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fa1900f0fa0 con 0x7fa198100000 2024-09-05T01:25:38.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa19e44c640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 0x7fa1840a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 -- start start 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48baf0 0x555a0d4905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d490b80 0x555a0d493040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a0d3e62f0 con 0x555a0d490b80 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a0d38e270 con 0x555a0d48baf0 2024-09-05T01:25:38.300 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.299+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a0d3e22b0 con 0x555a0d48b1b0 2024-09-05T01:25:38.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:35414/0 (socket says 172.21.15.129:35414) 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 -- 172.21.15.129:0/593950821 learned_addr learned my addr 172.21.15.129:0/593950821 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19e44c640 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d490b80 0x555a0d493040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19e44c640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 0x7fa1840a5360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1880095f0 tx=0x7fa188005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 -- 172.21.15.129:0/593950821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48baf0 msgr2=0x555a0d4905b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48baf0 0x555a0d4905b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 -- 172.21.15.129:0/593950821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d490b80 msgr2=0x555a0d493040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d490b80 0x555a0d493040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 -- 172.21.15.129:0/593950821 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a0d38d340 con 0x555a0d48b1b0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.300+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa190030670 tx=0x7fa1900bcf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c5307b72d3046f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.301+0000 7fa183fff640 1 -- 172.21.15.129:0/593950821 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900ee740 con 0x555a0d48b1b0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.301+0000 7fa183fff640 1 -- 172.21.15.129:0/593950821 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa190036040 con 0x555a0d48b1b0 2024-09-05T01:25:38.302 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.301+0000 7fa183fff640 1 -- 172.21.15.129:0/593950821 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19004d3a0 con 0x555a0d48b1b0 2024-09-05T01:25:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/593950821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 msgr2=0x555a0d48b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa190030670 tx=0x7fa1900bcf90 comp rx=0 tx=0).stop 2024-09-05T01:25:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/593950821 shutdown_connections 2024-09-05T01:25:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d490b80 0x555a0d493040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.303 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48baf0 0x555a0d4905b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/593950821 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d48b5b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/593950821 >> 172.21.15.129:0/593950821 conn(0x555a0d486820 msgr2=0x555a0d486c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/593950821 shutdown_connections 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.302+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/593950821 wait complete. 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 -- start start 2024-09-05T01:25:38.304 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 Processor -- start 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48b1b0 0x555a0d495030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48baf0 0x555a0d495570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a0d3e22b0 con 0x555a0d48b1b0 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555a0d36f290 con 0x555a0d490b80 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.303+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x555a0d493910 con 0x555a0d48baf0 2024-09-05T01:25:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa1a2c3f740 10 client.0 _ll_register_callbacks cb 0x555a0d3a0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19d44a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48baf0 0x555a0d495570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35182/0 (socket says 172.21.15.129:35182) 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 -- 172.21.15.129:0/2058360636 learned_addr learned my addr 172.21.15.129:0/2058360636 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 -- 172.21.15.129:0/2058360636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48baf0 msgr2=0x555a0d495570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48baf0 0x555a0d495570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 -- 172.21.15.129:0/2058360636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48b1b0 msgr2=0x555a0d495030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48b1b0 0x555a0d495030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.304+0000 7fa19e44c640 1 -- 172.21.15.129:0/2058360636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a0d0f9210 con 0x555a0d490b80 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.305+0000 7fa19e44c640 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa188012c40 tx=0x7fa188004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98c371842bbf00a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.305+0000 7fa1817fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188011cb0 con 0x555a0d490b80 2024-09-05T01:25:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa190036040 con 0x555a0d490b80 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fa188031440 con 0x555a0d490b80 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa188011e50 con 0x555a0d490b80 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1a2c3f740 10 client.25970 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a0d1c5620 con 0x555a0d490b80 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa190036040 con 0x555a0d490b80 2024-09-05T01:25:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.306+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa18803a690 con 0x555a0d490b80 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 10 client.25970 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 10 client.25970 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 20 client.25970 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d47f0c0 0x555a0d481520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 4 client.25970 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a0d3f3a20 con 0x555a0d47f0c0 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d47f0c0 0x555a0d481520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d47f0c0 0x555a0d481520 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a85987327a9090 server_cookie=de78ccaa9faa1f1f in_seq=0 out_seq=0 2024-09-05T01:25:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.307+0000 7fa1817fa640 10 client.25970 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1817fa640 1 -- 172.21.15.129:0/2058360636 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a0d3f3a20 con 0x555a0d47f0c0 2024-09-05T01:25:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1817fa640 10 client.25970 handle_command_reply: tid=0 2024-09-05T01:25:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 client.25970 shutdown 2024-09-05T01:25:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 msgr2=0x555a0d49c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa188012c40 tx=0x7fa188004730 comp rx=0 tx=0).stop 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 shutdown_connections 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d47f0c0 0x555a0d481520 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d49c5b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48baf0 0x555a0d495570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2058360636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48b1b0 0x555a0d495030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 >> 172.21.15.129:0/2058360636 conn(0x555a0d486820 msgr2=0x555a0d488030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.309+0000 7fa1817fa640 0 client.25970 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.310+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 shutdown_connections 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.310+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2058360636 wait complete. 2024-09-05T01:25:38.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.310+0000 7fa1a2c3f740 20 client.25970 trim_cache size 0 max 16384 2024-09-05T01:25:38.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.337+0000 7fa1a2c3f740 1 Processor -- start 2024-09-05T01:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.338+0000 7fa1a2c3f740 1 -- start start 2024-09-05T01:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d4876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d488150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.339 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981ab070 con 0x555a0d48baf0 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981a8940 con 0x555a0d490b80 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981a6790 con 0x555a0d48b1b0 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa19d44a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa19d44a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46726/0 (socket says 172.21.15.129:46726) 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.339+0000 7fa19d44a640 1 -- 172.21.15.129:0/2248722186 learned_addr learned my addr 172.21.15.129:0/2248722186 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:38.340 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 -- 172.21.15.129:0/2248722186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 msgr2=0x555a0d4876d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d4876d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 -- 172.21.15.129:0/2248722186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 msgr2=0x555a0d488150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d488150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 -- 172.21.15.129:0/2248722186 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a0d481ba0 con 0x555a0d48baf0 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.340+0000 7fa19d44a640 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa194009980 tx=0x7fa194009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db8576eec098069 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.341+0000 7fa1837fe640 1 -- 172.21.15.129:0/2248722186 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194011af0 con 0x555a0d48baf0 2024-09-05T01:25:38.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.341+0000 7fa1837fe640 1 -- 172.21.15.129:0/2248722186 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa194011c90 con 0x555a0d48baf0 2024-09-05T01:25:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.342+0000 7fa1837fe640 1 -- 172.21.15.129:0/2248722186 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19401d6f0 con 0x555a0d48baf0 2024-09-05T01:25:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.342+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2248722186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 msgr2=0x555a0d487c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.343 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.342+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa194009980 tx=0x7fa194009e40 comp rx=0 tx=0).stop 2024-09-05T01:25:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.346+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2248722186 shutdown_connections 2024-09-05T01:25:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.346+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d490b80 0x555a0d488150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.346+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d487c10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.346+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/2248722186 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d48b1b0 0x555a0d4876d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.346 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.346+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2248722186 >> 172.21.15.129:0/2248722186 conn(0x555a0d486820 msgr2=0x555a0d495030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.347+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2248722186 shutdown_connections 2024-09-05T01:25:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.348+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/2248722186 wait complete. 2024-09-05T01:25:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.348+0000 7fa1a2c3f740 1 -- start start 2024-09-05T01:25:38.349 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.348+0000 7fa1a2c3f740 1 Processor -- start 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d4a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d490b80 0x555a0d501cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981a6e00 con 0x555a0d48baf0 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981a8270 con 0x555a0d48b1b0 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1981a74d0 con 0x555a0d490b80 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.349+0000 7fa1a2c3f740 10 client.0 _ll_register_callbacks cb 0x7fa188011c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.355 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:35184/0 (socket says 172.21.15.129:35184) 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 -- 172.21.15.129:0/3415384163 learned_addr learned my addr 172.21.15.129:0/3415384163 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 -- 172.21.15.129:0/3415384163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d490b80 msgr2=0x555a0d501cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d490b80 0x555a0d501cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 -- 172.21.15.129:0/3415384163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 msgr2=0x555a0d4a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d4a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.351+0000 7fa19dc4b640 1 -- 172.21.15.129:0/3415384163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a0d5022f0 con 0x555a0d48b1b0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.352+0000 7fa19dc4b640 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1900bd330 tx=0x7fa190009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cde9cc6da93483db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.353+0000 7fa1817fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.353+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1900b9ca0 con 0x555a0d48b1b0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.353+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa194011c90 con 0x555a0d48b1b0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fa19004d3a0 con 0x555a0d48b1b0 2024-09-05T01:25:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa19004d5a0 con 0x555a0d48b1b0 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 10 client.25973 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555a0d5036d0 con 0x555a0d48b1b0 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa194011c90 con 0x555a0d48b1b0 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa190085070 con 0x555a0d48b1b0 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 10 client.25973 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 10 client.25973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 20 client.25973 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d504100 0x555a0d506520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 4 client.25973 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:25:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.354+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a0d3edd70 con 0x555a0d504100 2024-09-05T01:25:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.355+0000 7fa19d44a640 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d504100 0x555a0d506520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.357+0000 7fa19d44a640 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d504100 0x555a0d506520 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76092229cb211f76 server_cookie=37d597d3f0701bda in_seq=0 out_seq=0 2024-09-05T01:25:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.357+0000 7fa1817fa640 10 client.25973 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.378+0000 7fa1817fa640 1 -- 172.21.15.129:0/3415384163 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a0d3edd70 con 0x555a0d504100 2024-09-05T01:25:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.378+0000 7fa1817fa640 10 client.25973 handle_command_reply: tid=0 2024-09-05T01:25:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.378+0000 7fa1a2c3f740 1 client.25973 shutdown 2024-09-05T01:25:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 msgr2=0x555a0d4a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1900bd330 tx=0x7fa190009800 comp rx=0 tx=0).stop 2024-09-05T01:25:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 shutdown_connections 2024-09-05T01:25:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x555a0d504100 0x555a0d506520 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x555a0d490b80 0x555a0d501cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a0d48baf0 0x555a0d4a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 --2- 172.21.15.129:0/3415384163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555a0d48b1b0 0x555a0d4a4640 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1900bd330 tx=0x7fa190009800 comp rx=0 tx=0).stop 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 >> 172.21.15.129:0/3415384163 conn(0x555a0d486820 msgr2=0x555a0d494fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1817fa640 0 client.25973 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 shutdown_connections 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 1 -- 172.21.15.129:0/3415384163 wait complete. 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.379+0000 7fa1a2c3f740 20 client.25973 trim_cache size 0 max 16384 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:25:38.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:25:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.380+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 msgr2=0x7fa1840a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.380+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 0x7fa1840a5360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1880095f0 tx=0x7fa188005f90 comp rx=0 tx=0).stop 2024-09-05T01:25:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.381+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 msgr2=0x7fa198199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:38.382 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.381+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa19002aeb0 tx=0x7fa190003ba0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 shutdown_connections 2024-09-05T01:25:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1840a2ea0 0x7fa1840a5360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa198103c30 0x7fa1981a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa198101270 0x7fa19819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 --2- 172.21.15.129:0/1331427724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa198100000 0x7fa198199fe0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.385+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 >> 172.21.15.129:0/1331427724 conn(0x7fa1980fb500 msgr2=0x7fa198101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.387+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 shutdown_connections 2024-09-05T01:25:38.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:38.387+0000 7fa19feaf640 1 -- 172.21.15.129:0/1331427724 wait complete. 2024-09-05T01:25:39.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:39 smithi183 ceph-mon[34589]: pgmap v662: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:39.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:39 smithi129 ceph-mon[30979]: pgmap v662: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:39.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:39 smithi194 ceph-mon[34941]: pgmap v662: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 515.0 s, 994.1 tps, lat 100.299 ms stddev 41.721 2024-09-05T01:25:41.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:41 smithi183 ceph-mon[34589]: pgmap v663: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-09-05T01:25:41.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:41 smithi129 ceph-mon[30979]: pgmap v663: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-09-05T01:25:41.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:41 smithi194 ceph-mon[34941]: pgmap v663: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 915 op/s 2024-09-05T01:25:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:42 smithi183 ceph-mon[34589]: pgmap v664: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 974 op/s 2024-09-05T01:25:42.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:42 smithi129 ceph-mon[30979]: pgmap v664: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 974 op/s 2024-09-05T01:25:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:42 smithi194 ceph-mon[34941]: pgmap v664: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 8.7 MiB/s wr, 974 op/s 2024-09-05T01:25:45.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:45 smithi183 ceph-mon[34589]: pgmap v665: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 520.0 s, 997.0 tps, lat 100.331 ms stddev 36.117 2024-09-05T01:25:45.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:45 smithi129 ceph-mon[30979]: pgmap v665: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:45.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:45 smithi194 ceph-mon[34941]: pgmap v665: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:47.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:47 smithi183 ceph-mon[34589]: pgmap v666: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 901 op/s 2024-09-05T01:25:47.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:47 smithi129 ceph-mon[30979]: pgmap v666: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 901 op/s 2024-09-05T01:25:47.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:47 smithi194 ceph-mon[34941]: pgmap v666: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.9 MiB/s wr, 901 op/s 2024-09-05T01:25:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:48 smithi183 ceph-mon[34589]: pgmap v667: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:48.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:48 smithi129 ceph-mon[30979]: pgmap v667: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:48 smithi194 ceph-mon[34941]: pgmap v667: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 9.3 MiB/s wr, 1.05k op/s 2024-09-05T01:25:50.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 525.0 s, 978.8 tps, lat 102.090 ms stddev 35.511 2024-09-05T01:25:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:51 smithi183 ceph-mon[34589]: pgmap v668: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 897 op/s 2024-09-05T01:25:51.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:51 smithi129 ceph-mon[30979]: pgmap v668: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 897 op/s 2024-09-05T01:25:51.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:51 smithi194 ceph-mon[34941]: pgmap v668: 129 pgs: 129 active+clean; 1.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.8 MiB/s wr, 897 op/s 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 -- 172.21.15.129:0/3145811455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544108470 msgr2=0x7f6544108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 --2- 172.21.15.129:0/3145811455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544108470 0x7f6544108870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6534009930 tx=0x7f653402a420 comp rx=0 tx=0).stop 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 -- 172.21.15.129:0/3145811455 shutdown_connections 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 --2- 172.21.15.129:0/3145811455 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544109b40 0x7f65441103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 --2- 172.21.15.129:0/3145811455 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109180 0x7f6544109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 --2- 172.21.15.129:0/3145811455 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544108470 0x7f6544108870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 -- 172.21.15.129:0/3145811455 >> 172.21.15.129:0/3145811455 conn(0x7f6544077470 msgr2=0x7f65440798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 -- 172.21.15.129:0/3145811455 shutdown_connections 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.537+0000 7f6543fff640 1 -- 172.21.15.129:0/3145811455 wait complete. 2024-09-05T01:25:51.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.538+0000 7f6543fff640 1 Processor -- start 2024-09-05T01:25:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.538+0000 7f6543fff640 1 -- start start 2024-09-05T01:25:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544108470 0x7f654419e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109b40 0x7f65441a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6544113b30 con 0x7f6544109b40 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65441139b0 con 0x7f6544109180 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6543fff640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6544113cb0 con 0x7f6544108470 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42110/0 (socket says 172.21.15.129:42110) 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 -- 172.21.15.129:0/3717002182 learned_addr learned my addr 172.21.15.129:0/3717002182 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f6542ffd640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544108470 0x7f654419e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65437fe640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109b40 0x7f65441a57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 -- 172.21.15.129:0/3717002182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544108470 msgr2=0x7f654419e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544108470 0x7f654419e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 -- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109b40 msgr2=0x7f65441a57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109b40 0x7f65441a57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.539+0000 7f65427fc640 1 -- 172.21.15.129:0/3717002182 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654406bd40 con 0x7f6544109180 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f65427fc640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f653800e9f0 tx=0x7f653800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f652bfff640 1 -- 172.21.15.129:0/3717002182 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653800cd80 con 0x7f6544109180 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f652bfff640 1 -- 172.21.15.129:0/3717002182 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65380108e0 con 0x7f6544109180 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f652bfff640 1 -- 172.21.15.129:0/3717002182 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538018920 con 0x7f6544109180 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654406c030 con 0x7f6544109180 2024-09-05T01:25:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.540+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654406c530 con 0x7f6544109180 2024-09-05T01:25:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.544+0000 7f652bfff640 1 -- 172.21.15.129:0/3717002182 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6538010a90 con 0x7f6544109180 2024-09-05T01:25:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.545+0000 7f654c026740 1 Processor -- start 2024-09-05T01:25:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f652bfff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 0x7f652c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 -- start start 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f652bfff640 1 -- 172.21.15.129:0/3717002182 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f65380cf700 con 0x7f6544109180 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0e4b0 0x559520a12f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a13540 0x559520a15a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5595209712f0 con 0x559520a0e4b0 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559520919270 con 0x559520a0db70 2024-09-05T01:25:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f654c026740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55952096d2b0 con 0x559520a13540 2024-09-05T01:25:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42116/0 (socket says 172.21.15.129:42116) 2024-09-05T01:25:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f6542ffd640 1 -- 172.21.15.129:0/80861285 learned_addr learned my addr 172.21.15.129:0/80861285 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.546+0000 7f65437fe640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 0x7f652c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f65437fe640 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a13540 0x559520a15a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f65427fc640 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0e4b0 0x559520a12f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f6542ffd640 1 -- 172.21.15.129:0/80861285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a13540 msgr2=0x559520a15a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f6542ffd640 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a13540 0x559520a15a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f6542ffd640 1 -- 172.21.15.129:0/80861285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0e4b0 msgr2=0x559520a12f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f6542ffd640 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0e4b0 0x559520a12f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.547+0000 7f6542ffd640 1 -- 172.21.15.129:0/80861285 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559520918340 con 0x559520a0db70 2024-09-05T01:25:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.548+0000 7f6542ffd640 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f65340314e0 tx=0x7f6534004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99eba2379221b3f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.548+0000 7f65297fa640 1 -- 172.21.15.129:0/80861285 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534035030 con 0x559520a0db70 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.548+0000 7f65297fa640 1 -- 172.21.15.129:0/80861285 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534040070 con 0x559520a0db70 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.548+0000 7f65297fa640 1 -- 172.21.15.129:0/80861285 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65340318f0 con 0x559520a0db70 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 -- 172.21.15.129:0/80861285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 msgr2=0x559520a0df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f65437fe640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 0x7f652c0a5210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f653c005fd0 tx=0x7f653c009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f65340314e0 tx=0x7f6534004040 comp rx=0 tx=0).stop 2024-09-05T01:25:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 -- 172.21.15.129:0/80861285 shutdown_connections 2024-09-05T01:25:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a13540 0x559520a15a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0e4b0 0x559520a12f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 --2- 172.21.15.129:0/80861285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a0df70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.549+0000 7f654c026740 1 -- 172.21.15.129:0/80861285 >> 172.21.15.129:0/80861285 conn(0x559520a091e0 msgr2=0x559520a095e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.550+0000 7f654c026740 1 -- 172.21.15.129:0/80861285 shutdown_connections 2024-09-05T01:25:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.550+0000 7f654c026740 1 -- 172.21.15.129:0/80861285 wait complete. 2024-09-05T01:25:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.550+0000 7f654c026740 1 -- start start 2024-09-05T01:25:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 Processor -- start 2024-09-05T01:25:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a1f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559520a164b0 con 0x559520a13540 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5595208fa290 con 0x559520a0db70 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559520a16630 con 0x559520a0e4b0 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42124/0 (socket says 172.21.15.129:42124) 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f6542ffd640 1 -- 172.21.15.129:0/1911863561 learned_addr learned my addr 172.21.15.129:0/1911863561 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.551+0000 7f654c026740 10 client.0 _ll_register_callbacks cb 0x55952092b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f65427fc640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 -- 172.21.15.129:0/1911863561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 msgr2=0x559520a182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 -- 172.21.15.129:0/1911863561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 msgr2=0x559520a1f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a1f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 -- 172.21.15.129:0/1911863561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559520684210 con 0x559520a0db70 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.552+0000 7f6542ffd640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6534005ab0 tx=0x7f6534003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b52c5b2211995e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f6526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534035070 con 0x559520a0db70 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6534040070 con 0x559520a0db70 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f65340315f0 con 0x559520a0db70 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6534056050 con 0x559520a0db70 2024-09-05T01:25:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f654c026740 10 client.25985 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559520750620 con 0x559520a0db70 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.553+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6534040070 con 0x559520a0db70 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f65340458c0 con 0x559520a0db70 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 10 client.25985 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 20 client.25985 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a23a40 0x559520a25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 4 client.25985 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:25:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55952097ea20 con 0x559520a23a40 2024-09-05T01:25:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.554+0000 7f65427fc640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a23a40 0x559520a25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.556+0000 7f6526ff5640 10 client.25985 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.556+0000 7f65427fc640 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a23a40 0x559520a25f00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da26645c3477c87f server_cookie=8b84859f24b1579d in_seq=0 out_seq=0 2024-09-05T01:25:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f6526ff5640 1 -- 172.21.15.129:0/1911863561 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55952097ea20 con 0x559520a23a40 2024-09-05T01:25:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f6526ff5640 10 client.25985 handle_command_reply: tid=0 2024-09-05T01:25:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 client.25985 shutdown 2024-09-05T01:25:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 msgr2=0x559520a17d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6534005ab0 tx=0x7f6534003820 comp rx=0 tx=0).stop 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 shutdown_connections 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a23a40 0x559520a25f00 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a1f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.557+0000 7f654c026740 1 --2- 172.21.15.129:0/1911863561 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a17d60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.558+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 >> 172.21.15.129:0/1911863561 conn(0x559520a091e0 msgr2=0x559520a0a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.558+0000 7f6526ff5640 0 client.25985 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.558+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 shutdown_connections 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.558+0000 7f654c026740 1 -- 172.21.15.129:0/1911863561 wait complete. 2024-09-05T01:25:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.558+0000 7f654c026740 20 client.25985 trim_cache size 0 max 16384 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.586+0000 7f654c026740 1 Processor -- start 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f654c026740 1 -- start start 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f654c026740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f654c026740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f654c026740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.587+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a18220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42128/0 (socket says 172.21.15.129:42128) 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:59698/0 (socket says 172.21.15.129:59698) 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f654c026740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65441bbed0 con 0x559520a13540 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f654c026740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b97a0 con 0x559520a0db70 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f654c026740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b7640 con 0x559520a0e4b0 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65437fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f6542ffd640 1 -- 172.21.15.129:0/1435096177 learned_addr learned my addr 172.21.15.129:0/1435096177 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 -- 172.21.15.129:0/1435096177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 msgr2=0x559520a18220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a18220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 -- 172.21.15.129:0/1435096177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 msgr2=0x559520a095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.588+0000 7f65427fc640 1 -- 172.21.15.129:0/1435096177 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559520a0e990 con 0x559520a0e4b0 2024-09-05T01:25:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.589+0000 7f65427fc640 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f653800e9c0 tx=0x7f65380ccc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138687cbfc60faa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.589+0000 7f6528ff9640 1 -- 172.21.15.129:0/1435096177 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538099ea0 con 0x559520a0e4b0 2024-09-05T01:25:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.589+0000 7f6528ff9640 1 -- 172.21.15.129:0/1435096177 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65380ccea0 con 0x559520a0e4b0 2024-09-05T01:25:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.589+0000 7f6528ff9640 1 -- 172.21.15.129:0/1435096177 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653809b250 con 0x559520a0e4b0 2024-09-05T01:25:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- 172.21.15.129:0/1435096177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 msgr2=0x559520a1f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f653800e9c0 tx=0x7f65380ccc60 comp rx=0 tx=0).stop 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- 172.21.15.129:0/1435096177 shutdown_connections 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a13540 0x559520a095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1f2a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 --2- 172.21.15.129:0/1435096177 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a0db70 0x559520a18220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- 172.21.15.129:0/1435096177 >> 172.21.15.129:0/1435096177 conn(0x559520a091e0 msgr2=0x559520a17b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- 172.21.15.129:0/1435096177 shutdown_connections 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- 172.21.15.129:0/1435096177 wait complete. 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.590+0000 7f654c026740 1 -- start start 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 Processor -- start 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 0x559520a1f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.594 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 0x559520a1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b7c60 con 0x559520a0db70 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b90d0 con 0x559520a13540 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.591+0000 7f654c026740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f65441b8330 con 0x559520a0e4b0 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f654c026740 10 client.0 _ll_register_callbacks cb 0x7f6534009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 0x559520a1f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f6542ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 0x559520a1f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35614/0 (socket says 172.21.15.129:35614) 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f6542ffd640 1 -- 172.21.15.129:0/3011840523 learned_addr learned my addr 172.21.15.129:0/3011840523 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:25:51.595 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 0x559520a1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 -- 172.21.15.129:0/3011840523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 msgr2=0x559520a1bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 -- 172.21.15.129:0/3011840523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 msgr2=0x559520a1f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 0x559520a1f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.592+0000 7f65437fe640 1 -- 172.21.15.129:0/3011840523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559520a8c7d0 con 0x559520a13540 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.593+0000 7f65437fe640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 0x559520a1c130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f653c00f790 tx=0x7f653c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3ad01dad81ec6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.593+0000 7f6526ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.593+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c02e060 con 0x559520a13540 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.593+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f653c00e5c0 con 0x559520a13540 2024-09-05T01:25:51.597 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.593+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65380ccea0 con 0x559520a13540 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.594+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f653c01de40 con 0x559520a13540 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.594+0000 7f654c026740 10 client.25991 fetch_fsmap learned FSMap version 32 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.594+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559520a8dae0 con 0x559520a13540 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.594+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653c00ecb0 con 0x559520a13540 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.594+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f653c028070 con 0x559520a13540 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 10 client.25991 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 10 client.25991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 20 client.25991 populate_metadata read hostname 'smithi129' 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a8e4a0 0x559520a908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 4 client.25991 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:25:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f65380ccea0 con 0x559520a8e4a0 2024-09-05T01:25:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.595+0000 7f6542ffd640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a8e4a0 0x559520a908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:25:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.596+0000 7f6542ffd640 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a8e4a0 0x559520a908c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be2e068c86c0e9d server_cookie=6db38cd396f8a868 in_seq=0 out_seq=0 2024-09-05T01:25:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.596+0000 7f6526ff5640 10 client.25991 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.618+0000 7f6526ff5640 1 -- 172.21.15.129:0/3011840523 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f65380ccea0 con 0x559520a8e4a0 2024-09-05T01:25:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.618+0000 7f6526ff5640 10 client.25991 handle_command_reply: tid=0 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.618+0000 7f654c026740 1 client.25991 shutdown 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 msgr2=0x559520a1c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 0x559520a1c130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f653c00f790 tx=0x7f653c004ab0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 shutdown_connections 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559520a8e4a0 0x559520a908c0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559520a13540 0x559520a1c130 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f6526ff5640 0 client.25991 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559520a0e4b0 0x559520a1bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 --2- 172.21.15.129:0/3011840523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559520a0db70 0x559520a1f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 >> 172.21.15.129:0/3011840523 conn(0x559520a091e0 msgr2=0x559520a07a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 shutdown_connections 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 1 -- 172.21.15.129:0/3011840523 wait complete. 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.619+0000 7f654c026740 20 client.25991 trim_cache size 0 max 16384 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:25:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:25:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 msgr2=0x7f652c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 0x7f652c0a5210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f653c005fd0 tx=0x7f653c009bf0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 msgr2=0x7f654419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:25:51.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f653800e9f0 tx=0x7f653800eeb0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 shutdown_connections 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f652c0a2d50 0x7f652c0a5210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6544109b40 0x7f65441a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6544109180 0x7f654419e770 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 --2- 172.21.15.129:0/3717002182 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6544108470 0x7f654419e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 >> 172.21.15.129:0/3717002182 conn(0x7f6544077470 msgr2=0x7f654410c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 shutdown_connections 2024-09-05T01:25:51.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:25:51.620+0000 7f6543fff640 1 -- 172.21.15.129:0/3717002182 wait complete. 2024-09-05T01:25:52.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:52 smithi183 ceph-mon[34589]: pgmap v669: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 953 op/s 2024-09-05T01:25:52.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:52 smithi129 ceph-mon[30979]: pgmap v669: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 953 op/s 2024-09-05T01:25:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:52 smithi194 ceph-mon[34941]: pgmap v669: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.5 MiB/s wr, 953 op/s 2024-09-05T01:25:55.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 530.0 s, 915.6 tps, lat 108.947 ms stddev 55.035 2024-09-05T01:25:55.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:55 smithi183 ceph-mon[34589]: pgmap v670: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:55.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:55 smithi129 ceph-mon[30979]: pgmap v670: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:55.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:55 smithi194 ceph-mon[34941]: pgmap v670: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.0 MiB/s wr, 923 op/s 2024-09-05T01:25:56.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:56 smithi183 ceph-mon[34589]: pgmap v671: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 930 op/s 2024-09-05T01:25:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:56 smithi129 ceph-mon[30979]: pgmap v671: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 930 op/s 2024-09-05T01:25:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:56 smithi194 ceph-mon[34941]: pgmap v671: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 930 op/s 2024-09-05T01:25:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:25:58 smithi183 ceph-mon[34589]: pgmap v672: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 1.01k op/s 2024-09-05T01:25:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:25:58 smithi129 ceph-mon[30979]: pgmap v672: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 1.01k op/s 2024-09-05T01:25:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:25:58 smithi194 ceph-mon[34941]: pgmap v672: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 1.01k op/s 2024-09-05T01:26:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 535.0 s, 821.0 tps, lat 121.496 ms stddev 134.939 2024-09-05T01:26:01.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:01 smithi183 ceph-mon[34589]: pgmap v673: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 889 op/s 2024-09-05T01:26:01.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:01 smithi129 ceph-mon[30979]: pgmap v673: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 889 op/s 2024-09-05T01:26:01.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:01 smithi194 ceph-mon[34941]: pgmap v673: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.3 MiB/s wr, 889 op/s 2024-09-05T01:26:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:02 smithi183 ceph-mon[34589]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 972 op/s 2024-09-05T01:26:02.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:02 smithi129 ceph-mon[30979]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 972 op/s 2024-09-05T01:26:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:02 smithi194 ceph-mon[34941]: pgmap v674: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 11 MiB/s wr, 972 op/s 2024-09-05T01:26:04.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:04 smithi183 ceph-mon[34589]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 967 op/s 2024-09-05T01:26:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:04 smithi129 ceph-mon[30979]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 967 op/s 2024-09-05T01:26:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:04 smithi194 ceph-mon[34941]: pgmap v675: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 12 MiB/s wr, 967 op/s 2024-09-05T01:26:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.772+0000 7f8ca06f1640 1 -- 172.21.15.129:0/2767357907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 msgr2=0x7f8c98073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:04.774 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.772+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/2767357907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 0x7f8c98073180 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009ca0 tx=0x7f8c8c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.777+0000 7f8ca06f1640 1 -- 172.21.15.129:0/2767357907 shutdown_connections 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.777+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/2767357907 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c980736c0 0x7f8c98119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.777+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/2767357907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 0x7f8c98073180 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.777+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/2767357907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c98109700 0x7f8c98109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.777+0000 7f8ca06f1640 1 -- 172.21.15.129:0/2767357907 >> 172.21.15.129:0/2767357907 conn(0x7f8c9806ece0 msgr2=0x7f8c98071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:04.778 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.778+0000 7f8ca06f1640 1 -- 172.21.15.129:0/2767357907 shutdown_connections 2024-09-05T01:26:04.780 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.779+0000 7f8ca06f1640 1 -- 172.21.15.129:0/2767357907 wait complete. 2024-09-05T01:26:04.781 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.780+0000 7f8ca06f1640 1 Processor -- start 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 -- start start 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 0x7f8c981a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c980736c0 0x7f8c981a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9811c490 con 0x7f8c98072d20 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9811c310 con 0x7f8c980736c0 2024-09-05T01:26:04.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8ca06f1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9811c610 con 0x7f8c98109700 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.783+0000 7f8c9ec8e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39608/0 (socket says 172.21.15.129:39608) 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 -- 172.21.15.129:0/889221149 learned_addr learned my addr 172.21.15.129:0/889221149 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c980736c0 0x7f8c981a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 -- 172.21.15.129:0/889221149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c980736c0 msgr2=0x7f8c981a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c980736c0 0x7f8c981a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 -- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 msgr2=0x7f8c981a6ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 0x7f8c981a6ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.784+0000 7f8c9ec8e640 1 -- 172.21.15.129:0/889221149 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c981ae700 con 0x7f8c98109700 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d750 tx=0x7f8c9000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8c977fe640 1 -- 172.21.15.129:0/889221149 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90017070 con 0x7f8c98109700 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8c977fe640 1 -- 172.21.15.129:0/889221149 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c900054d0 con 0x7f8c98109700 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8c977fe640 1 -- 172.21.15.129:0/889221149 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004c70 con 0x7f8c98109700 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c981ae9f0 con 0x7f8c98109700 2024-09-05T01:26:04.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.785+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981aeef0 con 0x7f8c98109700 2024-09-05T01:26:04.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.789+0000 7f8ca348b740 1 Processor -- start 2024-09-05T01:26:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 -- start start 2024-09-05T01:26:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.791 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 0x558a27f5dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f60830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a27ebc2f0 con 0x558a27f5e370 2024-09-05T01:26:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a27e64270 con 0x558a27f592e0 2024-09-05T01:26:04.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8ca348b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558a27eb82b0 con 0x558a27f589a0 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8c9dc8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8c9dc8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39614/0 (socket says 172.21.15.129:39614) 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8c9dc8c640 1 -- 172.21.15.129:0/2894037406 learned_addr learned my addr 172.21.15.129:0/2894037406 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.791+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 0x558a27f5dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f60830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c977fe640 1 -- 172.21.15.129:0/889221149 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8c90004e10 con 0x7f8c98109700 2024-09-05T01:26:04.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 -- 172.21.15.129:0/2894037406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 msgr2=0x558a27f58da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 -- 172.21.15.129:0/2894037406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 msgr2=0x558a27f60830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f60830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 -- 172.21.15.129:0/2894037406 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a27e63340 con 0x558a27f592e0 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.792+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 0x558a27f5dda0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009ad0 tx=0x7f8c84009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740083f44bae0169 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.793+0000 7f8c94ff9640 1 -- 172.21.15.129:0/2894037406 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400ed10 con 0x558a27f592e0 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.793+0000 7f8c94ff9640 1 -- 172.21.15.129:0/2894037406 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c84005020 con 0x558a27f592e0 2024-09-05T01:26:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.793+0000 7f8c94ff9640 1 -- 172.21.15.129:0/2894037406 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84004a00 con 0x558a27f592e0 2024-09-05T01:26:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 -- 172.21.15.129:0/2894037406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 msgr2=0x558a27f5dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 0x558a27f5dda0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009ad0 tx=0x7f8c84009f90 comp rx=0 tx=0).stop 2024-09-05T01:26:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 -- 172.21.15.129:0/2894037406 shutdown_connections 2024-09-05T01:26:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f60830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f592e0 0x558a27f5dda0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2894037406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f58da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 -- 172.21.15.129:0/2894037406 >> 172.21.15.129:0/2894037406 conn(0x558a27f54010 msgr2=0x558a27f54410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 -- 172.21.15.129:0/2894037406 shutdown_connections 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8c977fe640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 0x7f8c880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.794+0000 7f8ca348b740 1 -- 172.21.15.129:0/2894037406 wait complete. 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8c977fe640 1 -- 172.21.15.129:0/889221149 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f8c900cee80 con 0x7f8c98109700 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 -- start start 2024-09-05T01:26:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 Processor -- start 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 0x558a27f62c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558a27f61390 con 0x558a27f589a0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558a27e45290 con 0x558a27f5e370 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558a27f61510 con 0x558a27f592e0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.795+0000 7f8ca348b740 10 client.0 _ll_register_callbacks cb 0x558a27e76140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 0x7f8c880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9ec8e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39620/0 (socket says 172.21.15.129:39620) 2024-09-05T01:26:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 -- 172.21.15.129:0/2729512620 learned_addr learned my addr 172.21.15.129:0/2729512620 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 0x558a27f62c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 -- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 msgr2=0x558a27f6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 -- 172.21.15.129:0/2729512620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 msgr2=0x558a27f62c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 0x558a27f62c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9dc8c640 1 -- 172.21.15.129:0/2729512620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a27bcf210 con 0x558a27f592e0 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.796+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 0x558a27f62c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.797+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009ca0 tx=0x7f8c8c032d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f6b934fa17f017 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.797+0000 7f8c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.797+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c003d80 con 0x558a27f592e0 2024-09-05T01:26:04.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.797+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f8c8c004040 con 0x558a27f592e0 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 0x7f8c880a5240 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009aa0 tx=0x7f8c8400ed50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c84005020 con 0x558a27f592e0 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c8c0098b0 con 0x558a27f592e0 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8ca348b740 10 client.25909 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a27c9b620 con 0x558a27f592e0 2024-09-05T01:26:04.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c84005020 con 0x558a27f592e0 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c8c030960 con 0x558a27f592e0 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.798+0000 7f8ca348b740 10 client.25909 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8ca348b740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8ca348b740 20 client.25909 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27f6eb40 0x558a27f70f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8ca348b740 4 client.25909 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a27ec9a20 con 0x558a27f6eb40 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.799+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27f6eb40 0x558a27f70f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.800+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27f6eb40 0x558a27f70f60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba51a8b524f27146 server_cookie=9fb03dd8d89a5b12 in_seq=0 out_seq=0 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.800+0000 7f8c81ffb640 10 client.25909 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.801+0000 7f8c81ffb640 1 -- 172.21.15.129:0/2729512620 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a27ec9a20 con 0x558a27f6eb40 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.801+0000 7f8c81ffb640 10 client.25909 handle_command_reply: tid=0 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.801+0000 7f8ca348b740 1 client.25909 shutdown 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 msgr2=0x558a27f63150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009ca0 tx=0x7f8c8c032d60 comp rx=0 tx=0).stop 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 shutdown_connections 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27f6eb40 0x558a27f70f60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.804 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f63150 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 --2- 172.21.15.129:0/2729512620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f589a0 0x558a27f62c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 >> 172.21.15.129:0/2729512620 conn(0x558a27f54010 msgr2=0x558a27f55820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8c81ffb640 0 client.25909 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 shutdown_connections 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 1 -- 172.21.15.129:0/2729512620 wait complete. 2024-09-05T01:26:04.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.802+0000 7f8ca348b740 20 client.25909 trim_cache size 0 max 16384 2024-09-05T01:26:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.828+0000 7f8ca348b740 1 Processor -- start 2024-09-05T01:26:04.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.828+0000 7f8ca348b740 1 -- start start 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f6a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f55400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981bbf30 con 0x558a27f5e370 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981b9800 con 0x558a27f589a0 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8ca348b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981b7650 con 0x558a27f592e0 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8c9e48d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8c9e48d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:48748/0 (socket says 172.21.15.129:48748) 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.829+0000 7f8c9e48d640 1 -- 172.21.15.129:0/722476778 learned_addr learned my addr 172.21.15.129:0/722476778 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f55400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f6a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 -- 172.21.15.129:0/722476778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 msgr2=0x558a27f6a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f6a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 -- 172.21.15.129:0/722476778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 msgr2=0x558a27f55400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f55400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 -- 172.21.15.129:0/722476778 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a27f61640 con 0x558a27f589a0 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f55400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:26:04.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.830+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009250 tx=0x7f8c8401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09e9f18d946dca6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.834+0000 7f8c83fff640 1 -- 172.21.15.129:0/722476778 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840053c0 con 0x558a27f589a0 2024-09-05T01:26:04.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.834+0000 7f8c83fff640 1 -- 172.21.15.129:0/722476778 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c84005560 con 0x558a27f589a0 2024-09-05T01:26:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.835+0000 7f8ca348b740 1 -- 172.21.15.129:0/722476778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 msgr2=0x558a27f54ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:04.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.835+0000 7f8ca348b740 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009250 tx=0x7f8c8401c000 comp rx=0 tx=0).stop 2024-09-05T01:26:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.836+0000 7f8ca348b740 1 -- 172.21.15.129:0/722476778 shutdown_connections 2024-09-05T01:26:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.836+0000 7f8ca348b740 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f5e370 0x558a27f55400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.836+0000 7f8ca348b740 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f592e0 0x558a27f6a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.836+0000 7f8ca348b740 1 --2- 172.21.15.129:0/722476778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f589a0 0x558a27f54ec0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.836+0000 7f8ca348b740 1 -- 172.21.15.129:0/722476778 >> 172.21.15.129:0/722476778 conn(0x558a27f54010 msgr2=0x558a27f62c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.838+0000 7f8ca348b740 1 -- 172.21.15.129:0/722476778 shutdown_connections 2024-09-05T01:26:04.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.839+0000 7f8ca348b740 1 -- 172.21.15.129:0/722476778 wait complete. 2024-09-05T01:26:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.839+0000 7f8ca348b740 1 -- start start 2024-09-05T01:26:04.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.839+0000 7f8ca348b740 1 Processor -- start 2024-09-05T01:26:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 0x558a27f67870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981b7cc0 con 0x558a27f592e0 2024-09-05T01:26:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981b9130 con 0x558a27f5e370 2024-09-05T01:26:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981b8390 con 0x558a27f589a0 2024-09-05T01:26:04.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8ca348b740 10 client.0 _ll_register_callbacks cb 0x7f8c8c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8c9e48d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8c9e48d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39640/0 (socket says 172.21.15.129:39640) 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8c9e48d640 1 -- 172.21.15.129:0/135433793 learned_addr learned my addr 172.21.15.129:0/135433793 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8c9ec8e640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.841+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 0x558a27f67870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 -- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 msgr2=0x558a27f68810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f68810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 -- 172.21.15.129:0/135433793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 msgr2=0x558a27f67870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 0x558a27f67870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 -- 172.21.15.129:0/135433793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a27f68e20 con 0x558a27f589a0 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 0x558a27f67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c9e48d640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005fd0 tx=0x7f8c840058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7220a67cb2a392cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84041070 con 0x558a27f589a0 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f8c84018870 con 0x558a27f589a0 2024-09-05T01:26:04.844 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.842+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c84005560 con 0x558a27f589a0 2024-09-05T01:26:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.843+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c84018a70 con 0x558a27f589a0 2024-09-05T01:26:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.843+0000 7f8ca348b740 10 client.25912 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:04.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.843+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a27f6a130 con 0x558a27f589a0 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.843+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c840328a0 con 0x558a27f589a0 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.843+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c84031030 con 0x558a27f589a0 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.844+0000 7f8ca348b740 10 client.25912 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8ca348b740 10 client.25912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8ca348b740 20 client.25912 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27fd9980 0x558a27fdbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8ca348b740 4 client.25912 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:26:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27fd9980 0x558a27fdbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.845+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8c84005560 con 0x558a27fd9980 2024-09-05T01:26:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.846+0000 7f8c9dc8c640 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27fd9980 0x558a27fdbd70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc530a058e25182a server_cookie=673d69e7c4f3c58f in_seq=0 out_seq=0 2024-09-05T01:26:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:04.846+0000 7f8c81ffb640 10 client.25912 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 540.0 s, 1011.0 tps, lat 90.451 ms stddev 49.355 2024-09-05T01:26:05.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.517+0000 7f8c81ffb640 1 -- 172.21.15.129:0/135433793 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8c84005560 con 0x558a27fd9980 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.517+0000 7f8c81ffb640 10 client.25912 handle_command_reply: tid=0 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 client.25912 shutdown 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 msgr2=0x558a27f67330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005fd0 tx=0x7f8c840058e0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 shutdown_connections 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558a27fd9980 0x558a27fdbd70 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558a27f5e370 0x558a27f68810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558a27f592e0 0x558a27f67870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 --2- 172.21.15.129:0/135433793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558a27f589a0 0x558a27f67330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 >> 172.21.15.129:0/135433793 conn(0x558a27f54010 msgr2=0x558a27f529a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:05.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8c81ffb640 0 client.25912 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 shutdown_connections 2024-09-05T01:26:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.518+0000 7f8ca348b740 1 -- 172.21.15.129:0/135433793 wait complete. 2024-09-05T01:26:05.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.519+0000 7f8ca348b740 20 client.25912 trim_cache size 0 max 16384 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.520+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 msgr2=0x7f8c880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.520+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 0x7f8c880a5240 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009aa0 tx=0x7f8c8400ed50 comp rx=0 tx=0).stop 2024-09-05T01:26:05.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.520+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 msgr2=0x7f8c981ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.520+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d750 tx=0x7f8c9000dc10 comp rx=0 tx=0).stop 2024-09-05T01:26:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 shutdown_connections 2024-09-05T01:26:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c880a2d80 0x7f8c880a5240 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c98109700 0x7f8c981ae090 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c980736c0 0x7f8c981a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 --2- 172.21.15.129:0/889221149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c98072d20 0x7f8c981a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 >> 172.21.15.129:0/889221149 conn(0x7f8c9806ece0 msgr2=0x7f8c98071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 shutdown_connections 2024-09-05T01:26:05.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:05.521+0000 7f8ca06f1640 1 -- 172.21.15.129:0/889221149 wait complete. 2024-09-05T01:26:07.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:07 smithi183 ceph-mon[34589]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 957 op/s 2024-09-05T01:26:07.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:07 smithi129 ceph-mon[30979]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 957 op/s 2024-09-05T01:26:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:07 smithi194 ceph-mon[34941]: pgmap v676: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 13 MiB/s wr, 957 op/s 2024-09-05T01:26:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:08 smithi183 ceph-mon[34589]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-09-05T01:26:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:08 smithi129 ceph-mon[30979]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-09-05T01:26:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:08 smithi194 ceph-mon[34941]: pgmap v677: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 16 MiB/s wr, 1.10k op/s 2024-09-05T01:26:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 545.0 s, 793.2 tps, lat 138.113 ms stddev 207.519 2024-09-05T01:26:11.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:11 smithi183 ceph-mon[34589]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-09-05T01:26:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:11 smithi129 ceph-mon[30979]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-09-05T01:26:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:11 smithi194 ceph-mon[34941]: pgmap v678: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 17 MiB/s wr, 1.06k op/s 2024-09-05T01:26:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:12 smithi183 ceph-mon[34589]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:12 smithi129 ceph-mon[30979]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:12 smithi194 ceph-mon[34941]: pgmap v679: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:14 smithi183 ceph-mon[34589]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:26:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:14 smithi129 ceph-mon[30979]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:26:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:14 smithi194 ceph-mon[34941]: pgmap v680: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 16 MiB/s wr, 1.01k op/s 2024-09-05T01:26:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 550.0 s, 958.6 tps, lat 103.937 ms stddev 68.972 2024-09-05T01:26:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:16 smithi183 ceph-mon[34589]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 15 MiB/s wr, 967 op/s 2024-09-05T01:26:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:16 smithi129 ceph-mon[30979]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 15 MiB/s wr, 967 op/s 2024-09-05T01:26:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:16 smithi194 ceph-mon[34941]: pgmap v681: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 15 MiB/s wr, 967 op/s 2024-09-05T01:26:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:18 smithi183 ceph-mon[34589]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-09-05T01:26:18.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:18 smithi129 ceph-mon[30979]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-09-05T01:26:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:18 smithi194 ceph-mon[34941]: pgmap v682: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 18 MiB/s wr, 1.08k op/s 2024-09-05T01:26:18.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 -- 172.21.15.129:0/3748144822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 msgr2=0x7fe3301095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3748144822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe3301095f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe320009900 tx=0x7fe32002a3f0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 -- 172.21.15.129:0/3748144822 shutdown_connections 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3748144822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3748144822 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe3301095f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3748144822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 0x7fe330108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 -- 172.21.15.129:0/3748144822 >> 172.21.15.129:0/3748144822 conn(0x7fe3300771d0 msgr2=0x7fe330079610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.676+0000 7fe32ffff640 1 -- 172.21.15.129:0/3748144822 shutdown_connections 2024-09-05T01:26:18.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.677+0000 7fe32ffff640 1 -- 172.21.15.129:0/3748144822 wait complete. 2024-09-05T01:26:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.677+0000 7fe32ffff640 1 Processor -- start 2024-09-05T01:26:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.678+0000 7fe32ffff640 1 -- start start 2024-09-05T01:26:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.678+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 0x7fe33019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.678+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe33019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32ffff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe330113b40 con 0x7fe330108410 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32ffff640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301139c0 con 0x7fe330109b30 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe33019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe33019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42208/0 (socket says 172.21.15.129:42208) 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43876/0 (socket says 172.21.15.129:43876) 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32e7fc640 1 -- 172.21.15.129:0/3375239155 learned_addr learned my addr 172.21.15.129:0/3375239155 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:18.680 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe330113cc0 con 0x7fe330109170 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.679+0000 7fe32effd640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 0x7fe33019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 -- 172.21.15.129:0/3375239155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 msgr2=0x7fe33019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe33019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 -- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 msgr2=0x7fe33019e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 0x7fe33019e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 -- 172.21.15.129:0/3375239155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3301a5df0 con 0x7fe330109b30 2024-09-05T01:26:18.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.680+0000 7fe32f7fe640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e970 tx=0x7fe32400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.683+0000 7fe317fff640 1 -- 172.21.15.129:0/3375239155 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32400ccc0 con 0x7fe330109b30 2024-09-05T01:26:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.683+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3301a60e0 con 0x7fe330109b30 2024-09-05T01:26:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.683+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3301a6630 con 0x7fe330109b30 2024-09-05T01:26:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.685+0000 7fe317fff640 1 -- 172.21.15.129:0/3375239155 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe324005020 con 0x7fe330109b30 2024-09-05T01:26:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.685+0000 7fe317fff640 1 -- 172.21.15.129:0/3375239155 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324004a00 con 0x7fe330109b30 2024-09-05T01:26:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.687+0000 7fe338050740 1 Processor -- start 2024-09-05T01:26:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe317fff640 1 -- 172.21.15.129:0/3375239155 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe324004ba0 con 0x7fe330109b30 2024-09-05T01:26:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 -- start start 2024-09-05T01:26:18.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd55b70 0x55c39cd55f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd5b540 0x55c39cd5da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c39ccb92f0 con 0x55c39cd55b70 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c39cc61270 con 0x55c39cd564b0 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.688+0000 7fe338050740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c39ccb52b0 con 0x55c39cd5b540 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43890/0 (socket says 172.21.15.129:43890) 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 -- 172.21.15.129:0/512021504 learned_addr learned my addr 172.21.15.129:0/512021504 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:18.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe317fff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 0x7fe3180a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 -- 172.21.15.129:0/512021504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd5b540 msgr2=0x55c39cd5da00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:26:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd5b540 0x55c39cd5da00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 -- 172.21.15.129:0/512021504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd55b70 msgr2=0x55c39cd55f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd55b70 0x55c39cd55f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe32effd640 1 -- 172.21.15.129:0/512021504 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39cc60340 con 0x55c39cd564b0 2024-09-05T01:26:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.689+0000 7fe317fff640 1 -- 172.21.15.129:0/3375239155 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fe3240cf4d0 con 0x7fe330109b30 2024-09-05T01:26:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.690+0000 7fe32effd640 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe3280103d0 tx=0x7fe328010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f674dab90a3dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.690+0000 7fe3157fa640 1 -- 172.21.15.129:0/512021504 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328011700 con 0x55c39cd564b0 2024-09-05T01:26:18.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.691+0000 7fe3157fa640 1 -- 172.21.15.129:0/512021504 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe328011d50 con 0x55c39cd564b0 2024-09-05T01:26:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.691+0000 7fe3157fa640 1 -- 172.21.15.129:0/512021504 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32800ee10 con 0x55c39cd564b0 2024-09-05T01:26:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.692+0000 7fe338050740 1 -- 172.21.15.129:0/512021504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 msgr2=0x55c39cd5af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.692+0000 7fe338050740 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe3280103d0 tx=0x7fe328010890 comp rx=0 tx=0).stop 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.693+0000 7fe32f7fe640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 0x7fe3180a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.693+0000 7fe338050740 1 -- 172.21.15.129:0/512021504 shutdown_connections 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.693+0000 7fe338050740 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd5b540 0x55c39cd5da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd5af70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 --2- 172.21.15.129:0/512021504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd55b70 0x55c39cd55f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 -- 172.21.15.129:0/512021504 >> 172.21.15.129:0/512021504 conn(0x55c39cd511e0 msgr2=0x55c39cd515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 -- 172.21.15.129:0/512021504 shutdown_connections 2024-09-05T01:26:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 -- 172.21.15.129:0/512021504 wait complete. 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.694+0000 7fe338050740 1 -- start start 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.695+0000 7fe32f7fe640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 0x7fe3180a5250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe32409c390 tx=0x7fe324004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.695+0000 7fe338050740 1 Processor -- start 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.695+0000 7fe338050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 0x55c39cd5fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe338050740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe338050740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd67420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe32effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 0x55c39cd5fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe32effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42226/0 (socket says 172.21.15.129:42226) 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 0x55c39cd5fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43896/0 (socket says 172.21.15.129:43896) 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe32effd640 1 -- 172.21.15.129:0/3603126032 learned_addr learned my addr 172.21.15.129:0/3603126032 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c39cd5e560 con 0x55c39cd5b540 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c39cc42290 con 0x55c39cd55b70 2024-09-05T01:26:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.696+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c39cd5e6e0 con 0x55c39cd564b0 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 -- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 msgr2=0x55c39cd5fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 0x55c39cd5fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 -- 172.21.15.129:0/3603126032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 msgr2=0x55c39cd67420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd67420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39c9cc210 con 0x55c39cd564b0 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe338050740 10 client.0 _ll_register_callbacks cb 0x55c39cc73140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe32effd640 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f80 tx=0x7fe32800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6daff546bf9db46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe312ff5640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.697+0000 7fe312ff5640 1 -- 172.21.15.129:0/3603126032 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328004c50 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.698+0000 7fe312ff5640 1 -- 172.21.15.129:0/3603126032 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fe328004df0 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.698+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe328011d50 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.698+0000 7fe312ff5640 1 -- 172.21.15.129:0/3603126032 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe328011500 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.698+0000 7fe338050740 10 client.25921 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.698+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55c39ca98620 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe312ff5640 1 -- 172.21.15.129:0/3603126032 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe3280116a0 con 0x55c39cd564b0 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 10 client.25921 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 20 client.25921 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cd6da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 4 client.25921 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe328011d50 con 0x55c39cd6b5f0 2024-09-05T01:26:18.702 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.699+0000 7fe32e7fc640 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cd6da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.700+0000 7fe32e7fc640 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cd6da60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b18b850db8ac67 server_cookie=4a8033fea3ab9615 in_seq=0 out_seq=0 2024-09-05T01:26:18.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.700+0000 7fe312ff5640 10 client.25921 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.701+0000 7fe312ff5640 1 -- 172.21.15.129:0/3603126032 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe328011d50 con 0x55c39cd6b5f0 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.701+0000 7fe312ff5640 10 client.25921 handle_command_reply: tid=0 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.701+0000 7fe338050740 1 client.25921 shutdown 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 msgr2=0x55c39cd603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe328000f80 tx=0x7fe32800edc0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 shutdown_connections 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cd6da60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd67420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd564b0 0x55c39cd603e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe338050740 1 --2- 172.21.15.129:0/3603126032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd55b70 0x55c39cd5fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.702+0000 7fe312ff5640 0 client.25921 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.703+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 >> 172.21.15.129:0/3603126032 conn(0x55c39cd511e0 msgr2=0x55c39cd529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.703+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 shutdown_connections 2024-09-05T01:26:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.703+0000 7fe338050740 1 -- 172.21.15.129:0/3603126032 wait complete. 2024-09-05T01:26:18.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.704+0000 7fe338050740 20 client.25921 trim_cache size 0 max 16384 2024-09-05T01:26:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.734+0000 7fe338050740 1 Processor -- start 2024-09-05T01:26:18.735 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.734+0000 7fe338050740 1 -- start start 2024-09-05T01:26:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe32e7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:42240/0 (socket says 172.21.15.129:42240) 2024-09-05T01:26:18.736 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe32e7fc640 1 -- 172.21.15.129:0/2258842786 learned_addr learned my addr 172.21.15.129:0/2258842786 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe32effd640 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd52b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c0125f0 con 0x55c39cd5b540 2024-09-05T01:26:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c00fec0 con 0x55c39cd564b0 2024-09-05T01:26:18.737 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.735+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c00dd10 con 0x55c39cd55b70 2024-09-05T01:26:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32e7fc640 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd52b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 -- 172.21.15.129:0/2258842786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 msgr2=0x55c39cd525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.738 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 -- 172.21.15.129:0/2258842786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 msgr2=0x55c39cd52b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd52b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 -- 172.21.15.129:0/2258842786 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c39cd5f9e0 con 0x55c39cd564b0 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.736+0000 7fe32effd640 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd673e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe328011cc0 tx=0x7fe328021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5a5fe86f377a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.737+0000 7fe314ff9640 1 -- 172.21.15.129:0/2258842786 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328010d90 con 0x55c39cd564b0 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.737+0000 7fe314ff9640 1 -- 172.21.15.129:0/2258842786 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe32800e8a0 con 0x55c39cd564b0 2024-09-05T01:26:18.739 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.737+0000 7fe314ff9640 1 -- 172.21.15.129:0/2258842786 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280054f0 con 0x55c39cd564b0 2024-09-05T01:26:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 msgr2=0x55c39cd673e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd673e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe328011cc0 tx=0x7fe328021000 comp rx=0 tx=0).stop 2024-09-05T01:26:18.740 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 shutdown_connections 2024-09-05T01:26:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd52b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd673e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.738+0000 7fe338050740 1 --2- 172.21.15.129:0/2258842786 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.741 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.739+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 >> 172.21.15.129:0/2258842786 conn(0x55c39cd511e0 msgr2=0x55c39cd502b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.741+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 shutdown_connections 2024-09-05T01:26:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.741+0000 7fe338050740 1 -- 172.21.15.129:0/2258842786 wait complete. 2024-09-05T01:26:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.741+0000 7fe338050740 1 -- start start 2024-09-05T01:26:18.742 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe338050740 1 Processor -- start 2024-09-05T01:26:18.743 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe338050740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe338050740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe338050740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd66260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe338050740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c00e380 con 0x55c39cd5b540 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe32effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe32effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43918/0 (socket says 172.21.15.129:43918) 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.742+0000 7fe32effd640 1 -- 172.21.15.129:0/3259784355 learned_addr learned my addr 172.21.15.129:0/3259784355 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c00f7f0 con 0x55c39cd564b0 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c00ea50 con 0x55c39cd55b70 2024-09-05T01:26:18.744 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe338050740 10 client.0 _ll_register_callbacks cb 0x7fe328009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 -- 172.21.15.129:0/3259784355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 msgr2=0x55c39cd64d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd64d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 -- 172.21.15.129:0/3259784355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 msgr2=0x55c39cd66260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd66260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39cd66870 con 0x55c39cd564b0 2024-09-05T01:26:18.745 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.743+0000 7fe32effd640 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe32800ef10 tx=0x7fe328005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f6d8ef585d44b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.745+0000 7fe312ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:26:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.745+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280116a0 con 0x55c39cd564b0 2024-09-05T01:26:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.745+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7fe328037070 con 0x55c39cd564b0 2024-09-05T01:26:18.746 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.745+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe32800e8a0 con 0x55c39cd564b0 2024-09-05T01:26:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe328011d50 con 0x55c39cd564b0 2024-09-05T01:26:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 10 client.26027 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c39cdd65b0 con 0x55c39cd564b0 2024-09-05T01:26:18.748 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe32800e8a0 con 0x55c39cd564b0 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe328004030 con 0x55c39cd564b0 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 10 client.26027 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 10 client.26027 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 20 client.26027 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cdd9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 4 client.26027 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.747+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c39ccc6a20 con 0x55c39cd6b5f0 2024-09-05T01:26:18.749 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.748+0000 7fe32e7fc640 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cdd9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.748+0000 7fe312ff5640 10 client.26027 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:18.750 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:18.749+0000 7fe32e7fc640 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cdd9020 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca54e415ed48278 server_cookie=432ee174e8ed14af in_seq=0 out_seq=0 2024-09-05T01:26:19.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:26:19.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:26:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:26:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 555.0 s, 920.4 tps, lat 109.044 ms stddev 56.398 2024-09-05T01:26:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:26:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:26:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:26:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:20 smithi183 ceph-mon[34589]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 15 MiB/s wr, 942 op/s 2024-09-05T01:26:20.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:26:20.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:26:20.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:26:20.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:20 smithi129 ceph-mon[30979]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 15 MiB/s wr, 942 op/s 2024-09-05T01:26:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:26:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:26:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:26:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:20 smithi194 ceph-mon[34941]: pgmap v683: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 15 MiB/s wr, 942 op/s 2024-09-05T01:26:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe312ff5640 1 -- 172.21.15.129:0/3259784355 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c39ccc6a20 con 0x55c39cd6b5f0 2024-09-05T01:26:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe312ff5640 10 client.26027 handle_command_reply: tid=0 2024-09-05T01:26:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 client.26027 shutdown 2024-09-05T01:26:20.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 msgr2=0x55c39cd65250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe32800ef10 tx=0x7fe328005270 comp rx=0 tx=0).stop 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 shutdown_connections 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c39cd6b5f0 0x55c39cdd9020 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c39cd5b540 0x55c39cd66260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c39cd564b0 0x55c39cd65250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 --2- 172.21.15.129:0/3259784355 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c39cd55b70 0x55c39cd64d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 >> 172.21.15.129:0/3259784355 conn(0x55c39cd511e0 msgr2=0x55c39cd515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe312ff5640 0 client.26027 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 shutdown_connections 2024-09-05T01:26:20.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 1 -- 172.21.15.129:0/3259784355 wait complete. 2024-09-05T01:26:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.835+0000 7fe338050740 20 client.26027 trim_cache size 0 max 16384 2024-09-05T01:26:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:26:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:26:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:26:20.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:26:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 msgr2=0x7fe3180a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 0x7fe3180a5250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe32409c390 tx=0x7fe324004730 comp rx=0 tx=0).stop 2024-09-05T01:26:20.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 msgr2=0x7fe3301a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe32400e970 tx=0x7fe32400ee30 comp rx=0 tx=0).stop 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 shutdown_connections 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe3180a2d90 0x7fe3180a5250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe330109b30 0x7fe3301a5780 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe330109170 0x7fe33019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 --2- 172.21.15.129:0/3375239155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe330108410 0x7fe33019e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 >> 172.21.15.129:0/3375239155 conn(0x7fe3300771d0 msgr2=0x7fe33010c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 shutdown_connections 2024-09-05T01:26:20.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:20.837+0000 7fe32ffff640 1 -- 172.21.15.129:0/3375239155 wait complete. 2024-09-05T01:26:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:22 smithi183 ceph-mon[34589]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 14 MiB/s wr, 900 op/s 2024-09-05T01:26:22.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:22 smithi129 ceph-mon[30979]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 14 MiB/s wr, 900 op/s 2024-09-05T01:26:22.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:22 smithi194 ceph-mon[34941]: pgmap v684: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 14 MiB/s wr, 900 op/s 2024-09-05T01:26:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:24 smithi183 ceph-mon[34589]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 989 op/s 2024-09-05T01:26:24.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:24 smithi129 ceph-mon[30979]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 989 op/s 2024-09-05T01:26:24.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:24 smithi194 ceph-mon[34941]: pgmap v685: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 17 MiB/s wr, 989 op/s 2024-09-05T01:26:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 560.0 s, 916.6 tps, lat 107.714 ms stddev 39.160 2024-09-05T01:26:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:26 smithi183 ceph-mon[34589]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-05T01:26:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:26 smithi129 ceph-mon[30979]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-05T01:26:26.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:26 smithi194 ceph-mon[34941]: pgmap v686: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 15 MiB/s wr, 917 op/s 2024-09-05T01:26:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:28 smithi183 ceph-mon[34589]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:28 smithi129 ceph-mon[30979]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:28 smithi194 ceph-mon[34941]: pgmap v687: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-09-05T01:26:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 565.0 s, 889.4 tps, lat 113.159 ms stddev 52.052 2024-09-05T01:26:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:30 smithi183 ceph-mon[34589]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-09-05T01:26:30.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:30 smithi129 ceph-mon[30979]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-09-05T01:26:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:30 smithi194 ceph-mon[34941]: pgmap v688: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 14 MiB/s wr, 958 op/s 2024-09-05T01:26:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:32 smithi183 ceph-mon[34589]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:26:32.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:32 smithi129 ceph-mon[30979]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:26:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:32 smithi194 ceph-mon[34941]: pgmap v689: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 14 MiB/s wr, 954 op/s 2024-09-05T01:26:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 -- 172.21.15.129:0/2727671748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 msgr2=0x7f262010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 --2- 172.21.15.129:0/2727671748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262010df10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2618009ca0 tx=0x7f261802a8b0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 -- 172.21.15.129:0/2727671748 shutdown_connections 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 --2- 172.21.15.129:0/2727671748 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f2620110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 --2- 172.21.15.129:0/2727671748 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262010df10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 --2- 172.21.15.129:0/2727671748 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 0x7f262010b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 -- 172.21.15.129:0/2727671748 >> 172.21.15.129:0/2727671748 conn(0x7f262006eb50 msgr2=0x7f2620070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 -- 172.21.15.129:0/2727671748 shutdown_connections 2024-09-05T01:26:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.967+0000 7f26258f7640 1 -- 172.21.15.129:0/2727671748 wait complete. 2024-09-05T01:26:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.968+0000 7f26258f7640 1 Processor -- start 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 -- start start 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 0x7f262019e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f26201a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113ae0 con 0x7f26201091f0 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113960 con 0x7f262010bb20 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26258f7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2620113c60 con 0x7f262010e450 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f261ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f261ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:36334/0 (socket says 172.21.15.129:36334) 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f261ffff640 1 -- 172.21.15.129:0/3175309784 learned_addr learned my addr 172.21.15.129:0/3175309784 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.969+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f26201a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26248f5640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 0x7f262019e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 -- 172.21.15.129:0/3175309784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 msgr2=0x7f262019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 -- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 msgr2=0x7f262019e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 0x7f262019e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 -- 172.21.15.129:0/3175309784 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26201a5e20 con 0x7f262010e450 2024-09-05T01:26:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.971+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f26201a57b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f261400da50 tx=0x7f261400df10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.974+0000 7f261dffb640 1 -- 172.21.15.129:0/3175309784 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400ada0 con 0x7f262010e450 2024-09-05T01:26:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.974+0000 7f261dffb640 1 -- 172.21.15.129:0/3175309784 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26140054d0 con 0x7f262010e450 2024-09-05T01:26:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.974+0000 7f261dffb640 1 -- 172.21.15.129:0/3175309784 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261400e6b0 con 0x7f262010e450 2024-09-05T01:26:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.974+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26201a6110 con 0x7f262010e450 2024-09-05T01:26:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.974+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a6610 con 0x7f262010e450 2024-09-05T01:26:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 Processor -- start 2024-09-05T01:26:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 -- start start 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2c33512f0 con 0x55b2c33ec4b0 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2c32f9270 con 0x55b2c33f1570 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.985+0000 7f26299d1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2c334d2b0 con 0x55b2c33ebb70 2024-09-05T01:26:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f261dffb640 1 -- 172.21.15.129:0/3175309784 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f261400e850 con 0x7f262010e450 2024-09-05T01:26:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:55842/0 (socket says 172.21.15.129:55842) 2024-09-05T01:26:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f261ffff640 1 -- 172.21.15.129:0/3921818652 learned_addr learned my addr 172.21.15.129:0/3921818652 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f261ffff640 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.994 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.986+0000 7f26248f5640 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261dffb640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 0x7f26080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261dffb640 1 -- 172.21.15.129:0/3175309784 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f26140d01d0 con 0x7f262010e450 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261ffff640 1 -- 172.21.15.129:0/3921818652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 msgr2=0x55b2c33f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261ffff640 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261ffff640 1 -- 172.21.15.129:0/3921818652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 msgr2=0x55b2c33f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261ffff640 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.987+0000 7f261ffff640 1 -- 172.21.15.129:0/3921818652 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c32f8340 con 0x55b2c33ebb70 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.988+0000 7f261ffff640 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2618009430 tx=0x7f2618009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c3ab9e576313d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.988+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 0x7f26080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.989+0000 7f26250f6640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 0x7f26080a5210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f261400bfd0 tx=0x7f2614010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.990+0000 7f2606ffd640 1 -- 172.21.15.129:0/3921818652 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618030ac0 con 0x55b2c33ebb70 2024-09-05T01:26:33.995 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.990+0000 7f2606ffd640 1 -- 172.21.15.129:0/3921818652 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2618036070 con 0x55b2c33ebb70 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.990+0000 7f2606ffd640 1 -- 172.21.15.129:0/3921818652 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004630 con 0x55b2c33ebb70 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 -- 172.21.15.129:0/3921818652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 msgr2=0x55b2c33ebf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2618009430 tx=0x7f2618009380 comp rx=0 tx=0).stop 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 -- 172.21.15.129:0/3921818652 shutdown_connections 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 --2- 172.21.15.129:0/3921818652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33ebf70 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.991+0000 7f26299d1740 1 -- 172.21.15.129:0/3921818652 >> 172.21.15.129:0/3921818652 conn(0x55b2c33e9160 msgr2=0x55b2c33e9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.992+0000 7f26299d1740 1 -- 172.21.15.129:0/3921818652 shutdown_connections 2024-09-05T01:26:33.996 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.992+0000 7f26299d1740 1 -- 172.21.15.129:0/3921818652 wait complete. 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.992+0000 7f26299d1740 1 -- start start 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.992+0000 7f26299d1740 1 Processor -- start 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2c33f4490 con 0x55b2c33ec4b0 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b2c32da290 con 0x55b2c33f1570 2024-09-05T01:26:33.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b2c33f4610 con 0x55b2c33ebb70 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f26299d1740 10 client.0 _ll_register_callbacks cb 0x55b2c330b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:55852/0 (socket says 172.21.15.129:55852) 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f261ffff640 1 -- 172.21.15.129:0/2863200574 learned_addr learned my addr 172.21.15.129:0/2863200574 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.993+0000 7f261ffff640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 -- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 msgr2=0x55b2c33ff350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 -- 172.21.15.129:0/2863200574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 msgr2=0x55b2c33f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 -- 172.21.15.129:0/2863200574 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c3064210 con 0x55b2c33ebb70 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.994+0000 7f261ffff640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2618009620 tx=0x7f2618009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827fe478aae405bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.995+0000 7f25fffff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.995+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26180046f0 con 0x55b2c33ebb70 2024-09-05T01:26:33.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.995+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618036070 con 0x55b2c33ebb70 2024-09-05T01:26:33.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:33.996+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f2618004b40 con 0x55b2c33ebb70 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.000+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2618004d40 con 0x55b2c33ebb70 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 10 client.25936 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2c3130620 con 0x55b2c33ebb70 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2618036070 con 0x55b2c33ebb70 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f261806b020 con 0x55b2c33ebb70 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 10 client.25936 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 20 client.25936 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:34.002 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c3403a90 0x55b2c3405ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 4 client.25936 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.001+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c335ea20 con 0x55b2c3403a90 2024-09-05T01:26:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.002+0000 7f26248f5640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c3403a90 0x55b2c3405ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.003 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.003+0000 7f25fffff640 10 client.25936 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.003+0000 7f26248f5640 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c3403a90 0x55b2c3405ef0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd7d37ee79ed79a server_cookie=b1cc4aaeef6f66 in_seq=0 out_seq=0 2024-09-05T01:26:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.004+0000 7f25fffff640 1 -- 172.21.15.129:0/2863200574 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2c335ea20 con 0x55b2c3403a90 2024-09-05T01:26:34.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.004+0000 7f25fffff640 10 client.25936 handle_command_reply: tid=0 2024-09-05T01:26:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.004+0000 7f26299d1740 1 client.25936 shutdown 2024-09-05T01:26:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.004+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 msgr2=0x55b2c33f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.004+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2618009620 tx=0x7f2618009960 comp rx=0 tx=0).stop 2024-09-05T01:26:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 shutdown_connections 2024-09-05T01:26:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c3403a90 0x55b2c3405ef0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 --2- 172.21.15.129:0/2863200574 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ebb70 0x55b2c33f7dd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.005+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 >> 172.21.15.129:0/2863200574 conn(0x55b2c33e9160 msgr2=0x55b2c33ea970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.006+0000 7f25fffff640 0 client.25936 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.006+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 shutdown_connections 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.006+0000 7f26299d1740 1 -- 172.21.15.129:0/2863200574 wait complete. 2024-09-05T01:26:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.006+0000 7f26299d1740 20 client.25936 trim_cache size 0 max 16384 2024-09-05T01:26:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.036+0000 7f26299d1740 1 Processor -- start 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 -- start start 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c33f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b3610 con 0x55b2c33ebb70 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b0ee0 con 0x55b2c33f1570 2024-09-05T01:26:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.038+0000 7f26299d1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26201aed30 con 0x55b2c33ec4b0 2024-09-05T01:26:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.040+0000 7f261ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.040+0000 7f261ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c33f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.040+0000 7f261ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c33f83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50010/0 (socket says 172.21.15.129:50010) 2024-09-05T01:26:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.040+0000 7f261ffff640 1 -- 172.21.15.129:0/1760107826 learned_addr learned my addr 172.21.15.129:0/1760107826 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 -- 172.21.15.129:0/1760107826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ec4b0 msgr2=0x55b2c33f0e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f0e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 -- 172.21.15.129:0/1760107826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 msgr2=0x55b2c33f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c33f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 -- 172.21.15.129:0/1760107826 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c33f4740 con 0x55b2c33f1570 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.041+0000 7f261ffff640 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26180056d0 tx=0x7f2618005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f70fe2d28e3459 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.042+0000 7f26067fc640 1 -- 172.21.15.129:0/1760107826 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004280 con 0x55b2c33f1570 2024-09-05T01:26:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.042+0000 7f26067fc640 1 -- 172.21.15.129:0/1760107826 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2618036040 con 0x55b2c33f1570 2024-09-05T01:26:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.042+0000 7f26067fc640 1 -- 172.21.15.129:0/1760107826 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618005a90 con 0x55b2c33f1570 2024-09-05T01:26:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.043+0000 7f26299d1740 1 -- 172.21.15.129:0/1760107826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 msgr2=0x55b2c33ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.043+0000 7f26299d1740 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f26180056d0 tx=0x7f2618005fd0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 -- 172.21.15.129:0/1760107826 shutdown_connections 2024-09-05T01:26:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33f1570 0x55b2c33ff270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33ec4b0 0x55b2c33f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 --2- 172.21.15.129:0/1760107826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c33f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 -- 172.21.15.129:0/1760107826 >> 172.21.15.129:0/1760107826 conn(0x55b2c33e9160 msgr2=0x55b2c33f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.044+0000 7f26299d1740 1 -- 172.21.15.129:0/1760107826 shutdown_connections 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.045+0000 7f26299d1740 1 -- 172.21.15.129:0/1760107826 wait complete. 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.045+0000 7f26299d1740 1 -- start start 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.045+0000 7f26299d1740 1 Processor -- start 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c340bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33ec4b0 0x55b2c33ff270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26201af3a0 con 0x55b2c33ebb70 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f26201b0810 con 0x55b2c33ec4b0 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f26201afa70 con 0x55b2c33f1570 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26299d1740 10 client.0 _ll_register_callbacks cb 0x7f2618030440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:55874/0 (socket says 172.21.15.129:55874) 2024-09-05T01:26:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 -- 172.21.15.129:0/3145235234 learned_addr learned my addr 172.21.15.129:0/3145235234 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f26248f5640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33ec4b0 0x55b2c33ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c340bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 -- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33ec4b0 msgr2=0x55b2c33ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.046+0000 7f261ffff640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33ec4b0 0x55b2c33ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f261ffff640 1 -- 172.21.15.129:0/3145235234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 msgr2=0x55b2c340bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f261ffff640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c340bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f261ffff640 1 -- 172.21.15.129:0/3145235234 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c346ce20 con 0x55b2c33f1570 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f261ffff640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b2c340ae90 tx=0x7f261802a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b630c4ac9ebfe76f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f25fffff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2618004b40 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9266+0+0 (secure 0 0 0) 0x7f2618004ee0 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.047+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2618036040 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f26180050e0 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 10 client.25939 fetch_fsmap learned FSMap version 32 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b2c346e080 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2618036040 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f261806b050 con 0x55b2c33f1570 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 10 client.25939 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 20 client.25939 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c346e920 0x55b2c3470d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 4 client.25939 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:26:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.048+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2c3358d70 con 0x55b2c346e920 2024-09-05T01:26:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.049+0000 7f26248f5640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c346e920 0x55b2c3470d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.050+0000 7f26248f5640 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c346e920 0x55b2c3470d40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ddd8ef3624a57c server_cookie=f61be7e5ebbf8911 in_seq=0 out_seq=0 2024-09-05T01:26:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.050+0000 7f25fffff640 10 client.25939 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.076+0000 7f25fffff640 1 -- 172.21.15.129:0/3145235234 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2c3358d70 con 0x55b2c346e920 2024-09-05T01:26:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f25fffff640 10 client.25939 handle_command_reply: tid=0 2024-09-05T01:26:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 client.25939 shutdown 2024-09-05T01:26:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 msgr2=0x55b2c346c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55b2c340ae90 tx=0x7f261802a8b0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 shutdown_connections 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b2c346e920 0x55b2c3470d40 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b2c33f1570 0x55b2c346c780 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b2c33ec4b0 0x55b2c33ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 --2- 172.21.15.129:0/3145235234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b2c33ebb70 0x55b2c340bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 >> 172.21.15.129:0/3145235234 conn(0x55b2c33e9160 msgr2=0x55b2c33f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f25fffff640 0 client.25939 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.077+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 shutdown_connections 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.078+0000 7f26299d1740 1 -- 172.21.15.129:0/3145235234 wait complete. 2024-09-05T01:26:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.078+0000 7f26299d1740 20 client.25939 trim_cache size 0 max 16384 2024-09-05T01:26:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:26:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:26:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:26:34.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 msgr2=0x7f26080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 0x7f26080a5210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f261400bfd0 tx=0x7f2614010040 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 msgr2=0x7f26201a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f26201a57b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f261400da50 tx=0x7f261400df10 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 shutdown_connections 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f26080a2d50 0x7f26080a5210 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f262010e450 0x7f26201a57b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f262010bb20 0x7f262019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 --2- 172.21.15.129:0/3175309784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26201091f0 0x7f262019e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.079+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 >> 172.21.15.129:0/3175309784 conn(0x7f262006eb50 msgr2=0x7f262010a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.080+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 shutdown_connections 2024-09-05T01:26:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:34.080+0000 7f26258f7640 1 -- 172.21.15.129:0/3175309784 wait complete. 2024-09-05T01:26:34.397 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:34 smithi194 ceph-mon[34941]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-05T01:26:34.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:34 smithi183 ceph-mon[34589]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-05T01:26:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:34 smithi129 ceph-mon[30979]: pgmap v690: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 14 MiB/s wr, 1.01k op/s 2024-09-05T01:26:34.750 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:26:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setuser ceph since I am not root 2024-09-05T01:26:34.751 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:26:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setgroup ceph since I am not root 2024-09-05T01:26:34.751 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:26:34 smithi194 ceph-mds[65377]: -- 172.21.15.194:0/1689760341 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55940b677860 con 0x55940b655400 2024-09-05T01:26:34.751 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:26:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: starting mds.c at 2024-09-05T01:26:35.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 570.0 s, 859.2 tps, lat 116.296 ms stddev 120.206 2024-09-05T01:26:35.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:35 smithi183 ceph-mon[34589]: daemon mds.c restarted 2024-09-05T01:26:35.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:35 smithi129 ceph-mon[30979]: daemon mds.c restarted 2024-09-05T01:26:35.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:35 smithi194 ceph-mon[34941]: daemon mds.c restarted 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6836/725078520,v1:172.21.15.194:6837/725078520] up:boot 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:26:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:36 smithi183 ceph-mon[34589]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 966 op/s 2024-09-05T01:26:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T01:26:36.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6836/725078520,v1:172.21.15.194:6837/725078520] up:boot 2024-09-05T01:26:36.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:26:36.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:26:36.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:26:36.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:36 smithi129 ceph-mon[30979]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 966 op/s 2024-09-05T01:26:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T01:26:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6836/725078520,v1:172.21.15.194:6837/725078520] up:boot 2024-09-05T01:26:36.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-09-05T01:26:36.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:26:36.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-05T01:26:36.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:36 smithi194 ceph-mon[34941]: pgmap v692: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 13 MiB/s wr, 966 op/s 2024-09-05T01:26:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:38 smithi183 ceph-mon[34589]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-05T01:26:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:38 smithi129 ceph-mon[30979]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-05T01:26:38.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:38 smithi194 ceph-mon[34941]: pgmap v693: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-05T01:26:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 575.0 s, 615.0 tps, lat 158.164 ms stddev 186.368 2024-09-05T01:26:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:40 smithi183 ceph-mon[34589]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-09-05T01:26:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:40 smithi129 ceph-mon[30979]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-09-05T01:26:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:40 smithi194 ceph-mon[34941]: pgmap v694: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 13 MiB/s wr, 987 op/s 2024-09-05T01:26:41.900 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T01:26:41.901 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:26:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:42 smithi129 ceph-mon[30979]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-09-05T01:26:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:26:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:42 smithi183 ceph-mon[34589]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-09-05T01:26:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:42 smithi194 ceph-mon[34941]: pgmap v695: 129 pgs: 129 active+clean; 1.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 13 MiB/s wr, 990 op/s 2024-09-05T01:26:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:44 smithi129 ceph-mon[30979]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 996 op/s 2024-09-05T01:26:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:44 smithi183 ceph-mon[34589]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 996 op/s 2024-09-05T01:26:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:44 smithi194 ceph-mon[34941]: pgmap v696: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 14 MiB/s wr, 996 op/s 2024-09-05T01:26:44.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.820+0000 7f6919594640 1 -- 172.21.15.129:0/3117327427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 msgr2=0x7f6914100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:44.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.820+0000 7f6919594640 1 --2- 172.21.15.129:0/3117327427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f6914100d60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f690c009ca0 tx=0x7f690c02a980 comp rx=0 tx=0).stop 2024-09-05T01:26:44.823 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.821+0000 7f6919594640 1 -- 172.21.15.129:0/3117327427 shutdown_connections 2024-09-05T01:26:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.821+0000 7f6919594640 1 --2- 172.21.15.129:0/3117327427 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f6914106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.821+0000 7f6919594640 1 --2- 172.21.15.129:0/3117327427 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f6914105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.821+0000 7f6919594640 1 --2- 172.21.15.129:0/3117327427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f6914100d60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:44.824 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.821+0000 7f6919594640 1 -- 172.21.15.129:0/3117327427 >> 172.21.15.129:0/3117327427 conn(0x7f69140fc610 msgr2=0x7f69140fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:44.825 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.824+0000 7f6919594640 1 -- 172.21.15.129:0/3117327427 shutdown_connections 2024-09-05T01:26:44.827 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.825+0000 7f6919594640 1 -- 172.21.15.129:0/3117327427 wait complete. 2024-09-05T01:26:44.829 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.826+0000 7f6919594640 1 Processor -- start 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.826+0000 7f6919594640 1 -- start start 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.827+0000 7f6919594640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f691419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.827+0000 7f6919594640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f691419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.827+0000 7f6919594640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f69141a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.827+0000 7f6919594640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6914118340 con 0x7f6914100980 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.828+0000 7f6919594640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69141181c0 con 0x7f69141012a0 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.827+0000 7f69137fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f691419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.828+0000 7f69137fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f691419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43180/0 (socket says 172.21.15.129:43180) 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.828+0000 7f69137fe640 1 -- 172.21.15.129:0/1811032312 learned_addr learned my addr 172.21.15.129:0/1811032312 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.828+0000 7f6918d93640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f69141a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:44.830 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.828+0000 7f6919594640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f69141184c0 con 0x7f6914105d90 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6913fff640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f691419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6918d93640 1 -- 172.21.15.129:0/1811032312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 msgr2=0x7f691419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6918d93640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f691419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6918d93640 1 -- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 msgr2=0x7f691419e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6918d93640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f691419e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:44.832 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.830+0000 7f6918d93640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69141a3630 con 0x7f6914105d90 2024-09-05T01:26:44.833 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.831+0000 7f6918d93640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f69141a2f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f690800b3e0 tx=0x7f690800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:44.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.833+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908015020 con 0x7f6914105d90 2024-09-05T01:26:44.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.834+0000 7f6919594640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69141a3920 con 0x7f6914105d90 2024-09-05T01:26:44.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.834+0000 7f6919594640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69141ab200 con 0x7f6914105d90 2024-09-05T01:26:44.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.835+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6908011ce0 con 0x7f6914105d90 2024-09-05T01:26:44.838 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.836+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908010e00 con 0x7f6914105d90 2024-09-05T01:26:44.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.841+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f691406a850 con 0x7f6914105d90 2024-09-05T01:26:44.843 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.842+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6908018020 con 0x7f6914105d90 2024-09-05T01:26:44.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.842+0000 7f69117fa640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 0x7f68fc0a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:44.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.843+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f69080cec50 con 0x7f6914105d90 2024-09-05T01:26:44.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.844+0000 7f6913fff640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 0x7f68fc0a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:44.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.847+0000 7f6913fff640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 0x7f68fc0a5000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f690c005fa0 tx=0x7f690c000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:44.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:44.848+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f690801f020 con 0x7f6914105d90 2024-09-05T01:26:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 580.0 s, 642.0 tps, lat 152.182 ms stddev 169.443 2024-09-05T01:26:45.258 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.256+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f691419f9e0 con 0x7f6914105d90 2024-09-05T01:26:45.259 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.257+0000 7f69117fa640 1 -- 172.21.15.129:0/1811032312 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f690809b740 con 0x7f6914105d90 2024-09-05T01:26:45.259 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:26:45.260 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:26:45.260 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:26:45.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.261+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 msgr2=0x7f68fc0a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:45.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.261+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 0x7f68fc0a5000 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f690c005fa0 tx=0x7f690c000f40 comp rx=0 tx=0).stop 2024-09-05T01:26:45.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.261+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 msgr2=0x7f69141a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:45.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.261+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f69141a2f50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f690800b3e0 tx=0x7f690800b8a0 comp rx=0 tx=0).stop 2024-09-05T01:26:45.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 shutdown_connections 2024-09-05T01:26:45.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f68fc0a2b40 0x7f68fc0a5000 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:45.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6914105d90 0x7f69141a2f50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:45.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69141012a0 0x7f691419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:45.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 --2- 172.21.15.129:0/1811032312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6914100980 0x7f691419e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:45.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 >> 172.21.15.129:0/1811032312 conn(0x7f69140fc610 msgr2=0x7f69140fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:45.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 shutdown_connections 2024-09-05T01:26:45.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:45.262+0000 7f68faffd640 1 -- 172.21.15.129:0/1811032312 wait complete. 2024-09-05T01:26:45.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:45 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1811032312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:45.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:45 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1811032312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:45.753 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:45 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1811032312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:46.182 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T01:26:46.182 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T01:26:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:46 smithi129 ceph-mon[30979]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-05T01:26:46.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:46 smithi183 ceph-mon[34589]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-05T01:26:46.668 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:26:46.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:46 smithi194 ceph-mon[34941]: pgmap v697: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 12 MiB/s wr, 906 op/s 2024-09-05T01:26:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.247+0000 7f9a18cf1640 1 -- 172.21.15.129:0/707206072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14101a60 msgr2=0x7f9a14101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 --2- 172.21.15.129:0/707206072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14101a60 0x7f9a14101ee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009900 tx=0x7f99fc02a410 comp rx=0 tx=0).stop 2024-09-05T01:26:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 -- 172.21.15.129:0/707206072 shutdown_connections 2024-09-05T01:26:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 --2- 172.21.15.129:0/707206072 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14102420 0x7f9a14108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 --2- 172.21.15.129:0/707206072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14101a60 0x7f9a14101ee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 --2- 172.21.15.129:0/707206072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 0x7f9a14100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.248+0000 7f9a18cf1640 1 -- 172.21.15.129:0/707206072 >> 172.21.15.129:0/707206072 conn(0x7f9a140fbfd0 msgr2=0x7f9a140fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.249+0000 7f9a18cf1640 1 -- 172.21.15.129:0/707206072 shutdown_connections 2024-09-05T01:26:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.249+0000 7f9a18cf1640 1 -- 172.21.15.129:0/707206072 wait complete. 2024-09-05T01:26:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.250+0000 7f9a18cf1640 1 Processor -- start 2024-09-05T01:26:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.251+0000 7f9a18cf1640 1 -- start start 2024-09-05T01:26:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.251+0000 7f9a18cf1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 0x7f9a141959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a18cf1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a18cf1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14102420 0x7f9a1419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:47472/0 (socket says 172.21.15.129:47472) 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2916737928 learned_addr learned my addr 172.21.15.129:0/2916737928 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14102420 0x7f9a1419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a18cf1640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1410b2e0 con 0x7f9a14100870 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a18cf1640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1410b160 con 0x7f9a14102420 2024-09-05T01:26:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.252+0000 7f9a18cf1640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a1410b460 con 0x7f9a14101a60 2024-09-05T01:26:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2916737928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14102420 msgr2=0x7f9a1419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14102420 0x7f9a1419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 msgr2=0x7f9a141959d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:26:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a1259c640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 0x7f9a141959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 0x7f9a141959d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.253+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a1419d5f0 con 0x7f9a14101a60 2024-09-05T01:26:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.254+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009230 tx=0x7f99fc004c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.254+0000 7f9a0bfff640 1 -- 172.21.15.129:0/2916737928 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc03f030 con 0x7f9a14101a60 2024-09-05T01:26:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.254+0000 7f9a18cf1640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a1419d880 con 0x7f9a14101a60 2024-09-05T01:26:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.255+0000 7f9a18cf1640 1 -- 172.21.15.129:0/2916737928 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a1419dcb0 con 0x7f9a14101a60 2024-09-05T01:26:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.258+0000 7f9a1ba90740 1 Processor -- start 2024-09-05T01:26:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.259+0000 7f9a0bfff640 1 -- 172.21.15.129:0/2916737928 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc0054b0 con 0x7f9a14101a60 2024-09-05T01:26:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.261+0000 7f9a0bfff640 1 -- 172.21.15.129:0/2916737928 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc035070 con 0x7f9a14101a60 2024-09-05T01:26:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.259+0000 7f9a1ba90740 1 -- start start 2024-09-05T01:26:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.261+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.261+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2539c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.261+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.261+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55705a1a92f0 con 0x55705a24ed10 2024-09-05T01:26:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55705a151270 con 0x55705a245a30 2024-09-05T01:26:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55705a1a52b0 con 0x55705a24e3d0 2024-09-05T01:26:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a0bfff640 1 -- 172.21.15.129:0/2916737928 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f99fc042030 con 0x7f9a14101a60 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43220/0 (socket says 172.21.15.129:43220) 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2902395963 learned_addr learned my addr 172.21.15.129:0/2902395963 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.262+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2902395963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 msgr2=0x55705a24e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2902395963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 msgr2=0x55705a2539c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:26:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a09d9b640 1 -- 172.21.15.129:0/2902395963 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705a150340 con 0x55705a245a30 2024-09-05T01:26:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.263+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.264+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0f5c20 tx=0x7f99fc0f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36207e00460abf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.264+0000 7f99fbfff640 1 -- 172.21.15.129:0/2902395963 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0fe070 con 0x55705a245a30 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.264+0000 7f99fbfff640 1 -- 172.21.15.129:0/2902395963 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc0f7d60 con 0x55705a245a30 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.264+0000 7f99fbfff640 1 -- 172.21.15.129:0/2902395963 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc0f9380 con 0x55705a245a30 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a0bfff640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 0x7f9a000a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 -- 172.21.15.129:0/2902395963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 msgr2=0x55705a247ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0f5c20 tx=0x7f99fc0f5d10 comp rx=0 tx=0).stop 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 -- 172.21.15.129:0/2902395963 shutdown_connections 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a245a30 0x55705a247ef0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/2902395963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.265+0000 7f9a1ba90740 1 -- 172.21.15.129:0/2902395963 >> 172.21.15.129:0/2902395963 conn(0x55705a241520 msgr2=0x55705a241920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.266+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 0x7f9a000a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.266+0000 7f9a1ba90740 1 -- 172.21.15.129:0/2902395963 shutdown_connections 2024-09-05T01:26:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.266+0000 7f9a0bfff640 1 -- 172.21.15.129:0/2916737928 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f99fc0ef9c0 con 0x7f9a14101a60 2024-09-05T01:26:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.267+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 0x7f9a000a5210 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005fd0 tx=0x7f9a0c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.267+0000 7f9a1ba90740 1 -- 172.21.15.129:0/2902395963 wait complete. 2024-09-05T01:26:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.268+0000 7f9a1ba90740 1 -- start start 2024-09-05T01:26:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.268+0000 7f9a1ba90740 1 Processor -- start 2024-09-05T01:26:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.268+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24ed10 0x55705a24d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55705a2486e0 con 0x55705a245a30 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55705a132290 con 0x55705a24ed10 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55705a248860 con 0x55705a24e3d0 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a1ba90740 10 client.0 _ll_register_callbacks cb 0x55705a163140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:47488/0 (socket says 172.21.15.129:47488) 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.269+0000 7f9a09d9b640 1 -- 172.21.15.129:0/1011339136 learned_addr learned my addr 172.21.15.129:0/1011339136 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24ed10 0x55705a24d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 -- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24ed10 msgr2=0x55705a24d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24ed10 0x55705a24d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 -- 172.21.15.129:0/1011339136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 msgr2=0x55705a249f10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:26:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 -- 172.21.15.129:0/1011339136 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557059ebc210 con 0x55705a24e3d0 2024-09-05T01:26:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.270+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0c5a00 tx=0x7f99fc0f7f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ed5f086e418959 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.271+0000 7f99f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.271+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc049030 con 0x55705a24e3d0 2024-09-05T01:26:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.272+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99fc0f7d60 con 0x55705a24e3d0 2024-09-05T01:26:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.272+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f99fc004250 con 0x55705a24e3d0 2024-09-05T01:26:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.273+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99fc0fe070 con 0x55705a24e3d0 2024-09-05T01:26:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.278+0000 7f9a1ba90740 10 client.25957 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.278+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557059f88620 con 0x55705a24e3d0 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc0f7d60 con 0x55705a24e3d0 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f99fc0b9930 con 0x55705a24e3d0 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 10 client.25957 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 10 client.25957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 20 client.25957 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a26c490 0x55705a26e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 4 client.25957 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.282+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55705a1b6a20 con 0x55705a26c490 2024-09-05T01:26:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.283+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a26c490 0x55705a26e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.283+0000 7f99f9ffb640 10 client.25957 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.286 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.284+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a26c490 0x55705a26e8c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d33c19a6d0865c4 server_cookie=fa5dce48dfb0cf0d in_seq=0 out_seq=0 2024-09-05T01:26:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.284+0000 7f99f9ffb640 1 -- 172.21.15.129:0/1011339136 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55705a1b6a20 con 0x55705a26c490 2024-09-05T01:26:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.284+0000 7f99f9ffb640 10 client.25957 handle_command_reply: tid=0 2024-09-05T01:26:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.285+0000 7f9a1ba90740 1 client.25957 shutdown 2024-09-05T01:26:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 msgr2=0x55705a24a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0c5a00 tx=0x7f99fc0f7f10 comp rx=0 tx=0).stop 2024-09-05T01:26:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 shutdown_connections 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a26c490 0x55705a26e8c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24ed10 0x55705a24d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1011339136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 >> 172.21.15.129:0/1011339136 conn(0x55705a241520 msgr2=0x55705a242d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.286+0000 7f99f9ffb640 0 client.25957 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.287+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 shutdown_connections 2024-09-05T01:26:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.287+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1011339136 wait complete. 2024-09-05T01:26:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.288+0000 7f9a1ba90740 20 client.25957 trim_cache size 0 max 16384 2024-09-05T01:26:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.320+0000 7f9a1ba90740 1 Processor -- start 2024-09-05T01:26:47.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.321+0000 7f9a1ba90740 1 -- start start 2024-09-05T01:26:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.321+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.321+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.321+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 0x55705a2423d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141aae90 con 0x55705a245a30 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a1ba90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141a8760 con 0x55705a24e3d0 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a12d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 0x55705a2423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a12d9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 0x55705a2423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:47494/0 (socket says 172.21.15.129:47494) 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a12d9d640 1 -- 172.21.15.129:0/1992220406 learned_addr learned my addr 172.21.15.129:0/1992220406 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141a65b0 con 0x55705a24ed10 2024-09-05T01:26:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.322+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.323+0000 7f9a1259c640 1 -- 172.21.15.129:0/1992220406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 msgr2=0x55705a2423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.323+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 0x55705a2423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.323+0000 7f9a1259c640 1 -- 172.21.15.129:0/1992220406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 msgr2=0x55705a24a450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:26:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.323+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.323+0000 7f9a1259c640 1 -- 172.21.15.129:0/1992220406 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55705a216700 con 0x55705a245a30 2024-09-05T01:26:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.324+0000 7f9a1259c640 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55705a26f1c0 tx=0x7f9a04005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75421c36b9b85d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.324+0000 7f99fb7fe640 1 -- 172.21.15.129:0/1992220406 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04009c40 con 0x55705a245a30 2024-09-05T01:26:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.324+0000 7f99fb7fe640 1 -- 172.21.15.129:0/1992220406 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a0401d030 con 0x55705a245a30 2024-09-05T01:26:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.325+0000 7f99fb7fe640 1 -- 172.21.15.129:0/1992220406 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04019250 con 0x55705a245a30 2024-09-05T01:26:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.325+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 msgr2=0x55705a249f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.325+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55705a26f1c0 tx=0x7f9a04005850 comp rx=0 tx=0).stop 2024-09-05T01:26:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.326+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 shutdown_connections 2024-09-05T01:26:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.326+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a24ed10 0x55705a2423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.326+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a24a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.326+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/1992220406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a245a30 0x55705a249f10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.326+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 >> 172.21.15.129:0/1992220406 conn(0x55705a241520 msgr2=0x55705a2500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.332+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 shutdown_connections 2024-09-05T01:26:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.332+0000 7f9a1ba90740 1 -- 172.21.15.129:0/1992220406 wait complete. 2024-09-05T01:26:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.332+0000 7f9a1ba90740 1 -- start start 2024-09-05T01:26:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1ba90740 1 Processor -- start 2024-09-05T01:26:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 0x55705a27d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1ba90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a27dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 0x55705a27d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1259c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 0x55705a27d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:47512/0 (socket says 172.21.15.129:47512) 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.333+0000 7f9a1259c640 1 -- 172.21.15.129:0/484296162 learned_addr learned my addr 172.21.15.129:0/484296162 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a27dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141a6c20 con 0x55705a24ed10 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141a8090 con 0x55705a24e3d0 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9a141a72f0 con 0x55705a245a30 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a1ba90740 10 client.0 _ll_register_callbacks cb 0x7f99fc0f5a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a12d9d640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 -- 172.21.15.129:0/484296162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 msgr2=0x55705a27d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 0x55705a27d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 -- 172.21.15.129:0/484296162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 msgr2=0x55705a2d5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2d5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55705a2d55a0 con 0x55705a24e3d0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.334+0000 7f9a09d9b640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a27dcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0b9720 tx=0x7f99fc005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4102b648ae1113bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.335+0000 7f99f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.335+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc049070 con 0x55705a24e3d0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.335+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f99fc0c5ca0 con 0x55705a24e3d0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.335+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a0401d030 con 0x55705a24e3d0 2024-09-05T01:26:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.335+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f99fc0bd090 con 0x55705a24e3d0 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.338+0000 7f9a1ba90740 10 client.26075 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.338+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55705a2d68b0 con 0x55705a24e3d0 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.338+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99fc0f5620 con 0x55705a24e3d0 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f99fc0bd520 con 0x55705a24e3d0 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 10 client.26075 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 10 client.26075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 20 client.26075 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a2d6de0 0x55705a2d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 4 client.26075 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:26:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9a0401d030 con 0x55705a2d6de0 2024-09-05T01:26:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f9a1259c640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a2d6de0 0x55705a2d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2549313896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a5180 msgr2=0x7f2fc00b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2549313896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a5180 0x7f2fc00b7040 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009900 tx=0x7f2fc402a480 comp rx=0 tx=0).stop 2024-09-05T01:26:47.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.339+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2549313896 shutdown_connections 2024-09-05T01:26:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.340+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2549313896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00b7580 0x7f2fc00b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.340+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2549313896 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a5180 0x7f2fc00b7040 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.345 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.340+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2549313896 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00a4860 0x7f2fc00a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.340+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2549313896 >> 172.21.15.129:0/2549313896 conn(0x7f2fc001a120 msgr2=0x7f2fc001a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.344+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2549313896 shutdown_connections 2024-09-05T01:26:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.344+0000 7f9a1259c640 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a2d6de0 0x55705a2d9200 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcea42b6fc12d22 server_cookie=e76bbd73071a5bc2 in_seq=0 out_seq=0 2024-09-05T01:26:47.347 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.344+0000 7f99f97fa640 10 client.26075 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.346+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2549313896 wait complete. 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.347+0000 7f2fcfffd640 1 Processor -- start 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.347+0000 7f2fcfffd640 1 -- start start 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.347+0000 7f2fcfffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 0x7f2fc0143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcfffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00a5180 0x7f2fc0143960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcfffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcfffd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00bcea0 con 0x7f2fc00a5180 2024-09-05T01:26:47.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fce59a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fce59a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43264/0 (socket says 172.21.15.129:43264) 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fce59a640 1 -- 172.21.15.129:0/2688960451 learned_addr learned my addr 172.21.15.129:0/2688960451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00bcd20 con 0x7f2fc00b7580 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00bd020 con 0x7f2fc00a4860 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 0x7f2fc0143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.348+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00a5180 0x7f2fc0143960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fce59a640 1 -- 172.21.15.129:0/2688960451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 msgr2=0x7f2fc0143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fce59a640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 0x7f2fc0143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fce59a640 1 -- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00a5180 msgr2=0x7f2fc0143960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fce59a640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00a5180 0x7f2fc0143960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fce59a640 1 -- 172.21.15.129:0/2688960451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc01483d0 con 0x7f2fc00b7580 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.349+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 0x7f2fc0143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.350+0000 7f2fce59a640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8051790 tx=0x7f2fc806feb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.350+0000 7f2fbeffd640 1 -- 172.21.15.129:0/2688960451 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc807e070 con 0x7f2fc00b7580 2024-09-05T01:26:47.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.350+0000 7f2fbeffd640 1 -- 172.21.15.129:0/2688960451 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc806f2e0 con 0x7f2fc00b7580 2024-09-05T01:26:47.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.350+0000 7f2fbeffd640 1 -- 172.21.15.129:0/2688960451 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80793f0 con 0x7f2fc00b7580 2024-09-05T01:26:47.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.350+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01486c0 con 0x7f2fc00b7580 2024-09-05T01:26:47.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.351+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc0148b20 con 0x7f2fc00b7580 2024-09-05T01:26:47.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.356+0000 7f2fbeffd640 1 -- 172.21.15.129:0/2688960451 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2fc8081070 con 0x7f2fc00b7580 2024-09-05T01:26:47.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.357+0000 7f2fd23d6740 1 Processor -- start 2024-09-05T01:26:47.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.357+0000 7f2fbeffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 0x7f2fb40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.358+0000 7f2fbeffd640 1 -- 172.21.15.129:0/2688960451 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fc81338a0 con 0x7f2fc00b7580 2024-09-05T01:26:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.358+0000 7f2fd23d6740 1 -- start start 2024-09-05T01:26:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.358+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1a8220 0x55d1ac1b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac10d2a0 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac0b4b10 con 0x55d1ac1a7900 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac109260 con 0x55d1ac1a8220 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43278/0 (socket says 172.21.15.129:43278) 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 -- 172.21.15.129:0/1097554451 learned_addr learned my addr 172.21.15.129:0/1097554451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 -- 172.21.15.129:0/1097554451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1a8220 msgr2=0x55d1ac1b25a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1a8220 0x55d1ac1b25a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 -- 172.21.15.129:0/1097554451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 msgr2=0x55d1ac1b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.359+0000 7f2fcdd99640 1 -- 172.21.15.129:0/1097554451 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1abe7eb90 con 0x55d1ac1a7900 2024-09-05T01:26:47.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.360+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009c30 tx=0x7f2fb8010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df6ec1e2c33eaa4b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.361+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 0x7f2fb40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.362+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 0x7f2fb40a5250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4000c00 tx=0x7f2fc4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.364+0000 7f2fb3fff640 1 -- 172.21.15.129:0/1097554451 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8004030 con 0x55d1ac1a7900 2024-09-05T01:26:47.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.364+0000 7f2fb3fff640 1 -- 172.21.15.129:0/1097554451 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fb80041d0 con 0x55d1ac1a7900 2024-09-05T01:26:47.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.365+0000 7f2fb3fff640 1 -- 172.21.15.129:0/1097554451 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80126b0 con 0x55d1ac1a7900 2024-09-05T01:26:47.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.365+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1097554451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 msgr2=0x55d1ac1a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.365+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009c30 tx=0x7f2fb8010740 comp rx=0 tx=0).stop 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1097554451 shutdown_connections 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1a8220 0x55d1ac1b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1097554451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a7900 0x55d1ac1a7ce0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1097554451 >> 172.21.15.129:0/1097554451 conn(0x55d1ac1a3e90 msgr2=0x55d1ac1a4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1097554451 shutdown_connections 2024-09-05T01:26:47.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.366+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1097554451 wait complete. 2024-09-05T01:26:47.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 -- start start 2024-09-05T01:26:47.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 Processor -- start 2024-09-05T01:26:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f99f97fa640 1 -- 172.21.15.129:0/484296162 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f9a0401d030 con 0x55705a2d6de0 2024-09-05T01:26:47.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f99f97fa640 10 client.26075 handle_command_reply: tid=0 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1afee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac1c6710 con 0x55d1ac1a7900 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac0956e0 con 0x55d1ac1a8220 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.367+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d1ac1c6890 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fd23d6740 10 client.0 _ll_register_callbacks cb 0x55d1ac0b9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:47.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f9a1ba90740 1 client.26075 shutdown 2024-09-05T01:26:47.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:47536/0 (socket says 172.21.15.129:47536) 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 -- 172.21.15.129:0/1758450864 learned_addr learned my addr 172.21.15.129:0/1758450864 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fcd598640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 -- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 msgr2=0x55d1ac1afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 -- 172.21.15.129:0/1758450864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 msgr2=0x55d1ac1af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.368+0000 7f2fce59a640 1 -- 172.21.15.129:0/1758450864 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ac1cfc80 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.369+0000 7f2fce59a640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80680a0 tx=0x7f2fc80ff5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d889d60720745ef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.369+0000 7f2fb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.369+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80ffc90 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.369+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb80041d0 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.369+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fc80fb950 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 msgr2=0x55705a27dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a27dcb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99fc0b9720 tx=0x7f99fc005660 comp rx=0 tx=0).stop 2024-09-05T01:26:47.373 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 shutdown_connections 2024-09-05T01:26:47.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fc8104040 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55705a2d6de0 0x55705a2d9200 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f99f97fa640 0 client.26075 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55705a24ed10 0x55705a2d5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55705a24e3d0 0x55705a27dcb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 --2- 172.21.15.129:0/484296162 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55705a245a30 0x55705a27d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f2fd23d6740 10 client.25969 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:47.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1ac1d2610 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.375 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.370+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 >> 172.21.15.129:0/484296162 conn(0x55705a241520 msgr2=0x55705a249cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 shutdown_connections 2024-09-05T01:26:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f9a1ba90740 1 -- 172.21.15.129:0/484296162 wait complete. 2024-09-05T01:26:47.376 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f9a1ba90740 20 client.26075 trim_cache size 0 max 16384 2024-09-05T01:26:47.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fb80041d0 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fc8078030 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:26:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:26:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:26:47.377 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:26:47.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 10 client.25969 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:47.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 10 client.25969 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:26:47.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 20 client.25969 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:47.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac1d3f50 0x55d1ac1d63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 4 client.25969 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:47.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1ac11a9d0 con 0x55d1ac1d3f50 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 msgr2=0x7f9a000a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 0x7f9a000a5210 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005fd0 tx=0x7f9a0c009500 comp rx=0 tx=0).stop 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 msgr2=0x7f9a14195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009230 tx=0x7f99fc004c00 comp rx=0 tx=0).stop 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 shutdown_connections 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9a000a2d50 0x7f9a000a5210 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9a14102420 0x7f9a1419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9a14101a60 0x7f9a14195f10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 --2- 172.21.15.129:0/2916737928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a14100870 0x7f9a141959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 >> 172.21.15.129:0/2916737928 conn(0x7f9a140fbfd0 msgr2=0x7f9a14104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.371+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac1d3f50 0x55d1ac1d63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 shutdown_connections 2024-09-05T01:26:47.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:26:47.372+0000 7f9a0959a640 1 -- 172.21.15.129:0/2916737928 wait complete. 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.374+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac1d3f50 0x55d1ac1d63f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d5ed237e321af9 server_cookie=5da0f3c76d46f4ce in_seq=0 out_seq=0 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.374+0000 7f2fb17fa640 10 client.25969 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.375+0000 7f2fb17fa640 1 -- 172.21.15.129:0/1758450864 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1ac11a9d0 con 0x55d1ac1d3f50 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.375+0000 7f2fb17fa640 10 client.25969 handle_command_reply: tid=0 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.375+0000 7f2fd23d6740 1 client.25969 shutdown 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 msgr2=0x55d1ac1b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80680a0 tx=0x7f2fc80ff5f0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 shutdown_connections 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac1d3f50 0x55d1ac1d63f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1b0420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/1758450864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 >> 172.21.15.129:0/1758450864 conn(0x55d1ac1a3e90 msgr2=0x55d1ac1a3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fb17fa640 0 client.25969 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 shutdown_connections 2024-09-05T01:26:47.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 1 -- 172.21.15.129:0/1758450864 wait complete. 2024-09-05T01:26:47.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.376+0000 7f2fd23d6740 20 client.25969 trim_cache size 0 max 16384 2024-09-05T01:26:47.411 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.409+0000 7f2fd23d6740 1 Processor -- start 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.410+0000 7f2fd23d6740 1 -- start start 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.410+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1b0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.410+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.410+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.410+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac012610 con 0x55d1ac1a7900 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac00fee0 con 0x55d1ac1a8220 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac00dd30 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcdd99640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.413 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43298/0 (socket says 172.21.15.129:43298) 2024-09-05T01:26:47.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 -- 172.21.15.129:0/2840637099 learned_addr learned my addr 172.21.15.129:0/2840637099 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.414 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fce59a640 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 -- 172.21.15.129:0/2840637099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 msgr2=0x55d1ac1dc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1dc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 -- 172.21.15.129:0/2840637099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 msgr2=0x55d1ac1b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.411+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.412+0000 7f2fcd598640 1 -- 172.21.15.129:0/2840637099 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1ac1a8050 con 0x55d1ac1a8220 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.412+0000 7f2fcd598640 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009900 tx=0x7f2fc403c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb3d4c4c7e02444 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.412+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2840637099 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc4035050 con 0x55d1ac1a8220 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.412+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2840637099 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc4004e30 con 0x55d1ac1a8220 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2840637099 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc40324b0 con 0x55d1ac1a8220 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 -- 172.21.15.129:0/2840637099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 msgr2=0x55d1ac1dbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009900 tx=0x7f2fc403c000 comp rx=0 tx=0).stop 2024-09-05T01:26:47.418 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 -- 172.21.15.129:0/2840637099 shutdown_connections 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1dc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1dbd00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.413+0000 7f2fd23d6740 1 -- 172.21.15.129:0/2840637099 >> 172.21.15.129:0/2840637099 conn(0x55d1ac1a3e90 msgr2=0x55d1ac1b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.414+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/2840637099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1a7900 0x55d1ac1b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.414+0000 7f2fd23d6740 1 -- 172.21.15.129:0/2840637099 shutdown_connections 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.414+0000 7f2fd23d6740 1 -- 172.21.15.129:0/2840637099 wait complete. 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.414+0000 7f2fd23d6740 1 -- start start 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.414+0000 7f2fd23d6740 1 Processor -- start 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1acb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 0x55d1ac23dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.419 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac00e3a0 con 0x55d1ac1b2ae0 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac00f810 con 0x55d1ac1a8220 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac00ea70 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.415+0000 7f2fd23d6740 10 client.0 _ll_register_callbacks cb 0x7f2fc80ffad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fcdd99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fcdd99640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1acb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:43306/0 (socket says 172.21.15.129:43306) 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fcdd99640 1 -- 172.21.15.129:0/203967293 learned_addr learned my addr 172.21.15.129:0/203967293 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:47.420 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fce59a640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 0x55d1ac23dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fcd598640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fce59a640 1 -- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 msgr2=0x55d1ac1acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.416+0000 7f2fce59a640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fce59a640 1 -- 172.21.15.129:0/203967293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 msgr2=0x55d1ac1ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fce59a640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fce59a640 1 -- 172.21.15.129:0/203967293 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ac23e2c0 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fce59a640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 0x55d1ac23dcd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80fc5d0 tx=0x7f2fc8068300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b38abaa2639ddfd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.417+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc806bcc0 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.421 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.418+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fc4004e30 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.418+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fc8108b50 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.422 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.418+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fc80feb50 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.418+0000 7f2fd23d6740 10 client.25975 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:47.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.418+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d1ac23f580 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.423 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.421+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fc4004e30 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fc8078040 con 0x55d1ac1ad5e0 2024-09-05T01:26:47.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 10 client.25975 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:47.424 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 10 client.25975 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 20 client.25975 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac23fd80 0x55d1ac2421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 4 client.25975 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.422+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55d1ac114d20 con 0x55d1ac23fd80 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.423+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac23fd80 0x55d1ac2421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:47.425 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.423+0000 7f2fcdd99640 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac23fd80 0x55d1ac2421a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a6ba9e13b22a20 server_cookie=1d03992fba6e374a in_seq=0 out_seq=0 2024-09-05T01:26:47.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.423+0000 7f2fb17fa640 10 client.25975 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.800 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.797+0000 7f2fb17fa640 1 -- 172.21.15.129:0/203967293 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1ac114d20 con 0x55d1ac23fd80 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.797+0000 7f2fb17fa640 10 client.25975 handle_command_reply: tid=0 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.797+0000 7f2fd23d6740 1 client.25975 shutdown 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 msgr2=0x55d1ac23dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 0x55d1ac23dcd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2fc80fc5d0 tx=0x7f2fc8068300 comp rx=0 tx=0).stop 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 shutdown_connections 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d1ac23fd80 0x55d1ac2421a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d1ac1ad5e0 0x55d1ac23dcd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fb17fa640 0 client.25975 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d1ac1b2ae0 0x55d1ac1ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 --2- 172.21.15.129:0/203967293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d1ac1a8220 0x55d1ac1acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.801 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 >> 172.21.15.129:0/203967293 conn(0x55d1ac1a3e90 msgr2=0x55d1ac1a9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 shutdown_connections 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 1 -- 172.21.15.129:0/203967293 wait complete. 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fd23d6740 20 client.25975 trim_cache size 0 max 16384 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 38, 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "fdf01480": { 2024-09-05T01:26:47.802 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 38, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:26:47.803 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 msgr2=0x7f2fb40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 0x7f2fb40a5250 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4000c00 tx=0x7f2fc4031400 comp rx=0 tx=0).stop 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 msgr2=0x7f2fc0147cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8051790 tx=0x7f2fc806feb0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 shutdown_connections 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fb40a2d90 0x7f2fb40a5250 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fc00b7580 0x7f2fc0147cf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fc00a5180 0x7f2fc0143960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.805 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 --2- 172.21.15.129:0/2688960451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fc00a4860 0x7f2fc0143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:47.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.798+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 >> 172.21.15.129:0/2688960451 conn(0x7f2fc001a120 msgr2=0x7f2fc00b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:47.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.799+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 shutdown_connections 2024-09-05T01:26:47.806 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:47.799+0000 7f2fcfffd640 1 -- 172.21.15.129:0/2688960451 wait complete. 2024-09-05T01:26:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:48 smithi183 ceph-mon[34589]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-05T01:26:48.482 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:48 smithi129 ceph-mon[30979]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-05T01:26:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:48 smithi194 ceph-mon[34941]: pgmap v698: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 14 MiB/s wr, 1.03k op/s 2024-09-05T01:26:48.511 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fdf01480', quiesce took 6.6 seconds. Will release after: 3.4000000000000004 2024-09-05T01:26:50.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 585.0 s, 311.0 tps, lat 158.294 ms stddev 166.063 2024-09-05T01:26:50.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:50 smithi183 ceph-mon[34589]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-05T01:26:50.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:50 smithi129 ceph-mon[30979]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-05T01:26:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:50 smithi194 ceph-mon[34941]: pgmap v699: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 12 MiB/s wr, 852 op/s 2024-09-05T01:26:51.912 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:26:52.174 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:26:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:52 smithi129 ceph-mon[30979]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 714 op/s 2024-09-05T01:26:52.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:52 smithi183 ceph-mon[34589]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 714 op/s 2024-09-05T01:26:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:52 smithi194 ceph-mon[34941]: pgmap v700: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 714 op/s 2024-09-05T01:26:52.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/1033368194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4106a80 msgr2=0x7fb0e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/1033368194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4106a80 0x7fb0e4106e60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009930 tx=0x7fb0d802a420 comp rx=0 tx=0).stop 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/1033368194 shutdown_connections 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/1033368194 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 0x7fb0e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/1033368194 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4100a80 0x7fb0e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/1033368194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4106a80 0x7fb0e4106e60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/1033368194 >> 172.21.15.129:0/1033368194 conn(0x7fb0e40fc7a0 msgr2=0x7fb0e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/1033368194 shutdown_connections 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.643+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/1033368194 wait complete. 2024-09-05T01:26:52.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 Processor -- start 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 -- start start 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 0x7fb0e4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4106a80 0x7fb0e419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e4111e50 con 0x7fb0e4100a80 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e4111cd0 con 0x7fb0e4106a80 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ec4c4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e4111fd0 con 0x7fb0e4101420 2024-09-05T01:26:52.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46588/0 (socket says 172.21.15.129:46588) 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 -- 172.21.15.129:0/752908667 learned_addr learned my addr 172.21.15.129:0/752908667 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0e9a5f640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 0x7fb0e4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0eaa61640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4106a80 0x7fb0e419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 -- 172.21.15.129:0/752908667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 msgr2=0x7fb0e4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 0x7fb0e4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 -- 172.21.15.129:0/752908667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4106a80 msgr2=0x7fb0e419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4106a80 0x7fb0e419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.644+0000 7fb0ea260640 1 -- 172.21.15.129:0/752908667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e419d3e0 con 0x7fb0e4100a80 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0ea260640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004770 tx=0x7fb0d80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d803f070 con 0x7fb0e4100a80 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb0d803b460 con 0x7fb0e4100a80 2024-09-05T01:26:52.674 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e419d670 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8035070 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.645+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e41a4fb0 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.646+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0e4100ee0 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.648+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb0d803b610 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.648+0000 7fb0d77fe640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 0x7fb0d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.649+0000 7fb0e9a5f640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 0x7fb0d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.649+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb0d80efa00 con 0x7fb0e4100a80 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.649+0000 7fb0e9a5f640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 0x7fb0d00a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40feec0 tx=0x7fb0e0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:52.675 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.652+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0d80bc550 con 0x7fb0e4100a80 2024-09-05T01:26:52.862 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.860+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0e4199790 con 0x7fb0e4100a80 2024-09-05T01:26:52.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.860+0000 7fb0d77fe640 1 -- 172.21.15.129:0/752908667 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7fb0e4199790 con 0x7fb0e4100a80 2024-09-05T01:26:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:26:52.864 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:26:52.864 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:26:52.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 msgr2=0x7fb0d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:52.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 0x7fb0d00a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40feec0 tx=0x7fb0e0009110 comp rx=0 tx=0).stop 2024-09-05T01:26:52.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 msgr2=0x7fb0e4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:52.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8004770 tx=0x7fb0d80042f0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 shutdown_connections 2024-09-05T01:26:52.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb0d00a2c00 0x7fb0d00a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb0e4106a80 0x7fb0e419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb0e4101420 0x7fb0e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 --2- 172.21.15.129:0/752908667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb0e4100a80 0x7fb0e4198430 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 >> 172.21.15.129:0/752908667 conn(0x7fb0e40fc7a0 msgr2=0x7fb0e40fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 shutdown_connections 2024-09-05T01:26:52.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:52.863+0000 7fb0ec4c4640 1 -- 172.21.15.129:0/752908667 wait complete. 2024-09-05T01:26:53.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:53 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/752908667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:53.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:53 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/752908667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:53.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:53 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/752908667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:26:53.573 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id fdf01480 --release --await' 2024-09-05T01:26:53.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id fdf01480 --release --await 2024-09-05T01:26:53.825 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:26:54.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:54 smithi129 ceph-mon[30979]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-09-05T01:26:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 -- 172.21.15.129:0/3550763207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc100ca0 msgr2=0x7f2fbc106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 --2- 172.21.15.129:0/3550763207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc106840 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009930 tx=0x7f2fa402a420 comp rx=0 tx=0).stop 2024-09-05T01:26:54.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 -- 172.21.15.129:0/3550763207 shutdown_connections 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 --2- 172.21.15.129:0/3550763207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc106f50 0x7f2fbc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 --2- 172.21.15.129:0/3550763207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc106840 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 --2- 172.21.15.129:0/3550763207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc1002f0 0x7f2fbc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 -- 172.21.15.129:0/3550763207 >> 172.21.15.129:0/3550763207 conn(0x7f2fbc077210 msgr2=0x7f2fbc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 -- 172.21.15.129:0/3550763207 shutdown_connections 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.335+0000 7f2fc085f640 1 -- 172.21.15.129:0/3550763207 wait complete. 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 Processor -- start 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 -- start start 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc1002f0 0x7f2fbc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc111e10 con 0x7f2fbc106f50 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc111c90 con 0x7f2fbc1002f0 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fc085f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc111f90 con 0x7f2fbc100ca0 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fbad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fbad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46608/0 (socket says 172.21.15.129:46608) 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fbad9d640 1 -- 172.21.15.129:0/2187559752 learned_addr learned my addr 172.21.15.129:0/2187559752 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.336+0000 7f2fba59c640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc1002f0 0x7f2fbc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 -- 172.21.15.129:0/2187559752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc100ca0 msgr2=0x7f2fbc198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 -- 172.21.15.129:0/2187559752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc1002f0 msgr2=0x7f2fbc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc1002f0 0x7f2fbc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 -- 172.21.15.129:0/2187559752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc19d300 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fbad9d640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400eaa0 tx=0x7f2fb400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2187559752 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4004d20 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fbc19d5f0 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2187559752 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fb4005020 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2187559752 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40116b0 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.337+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fbc1a4ed0 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.339+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2187559752 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2fb40051d0 con 0x7f2fbc106f50 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 Processor -- start 2024-09-05T01:26:54.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 -- start start 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9e9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ea6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587d9e01620 con 0x5587d9ea7240 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587d9da8e50 con 0x5587d9e9c060 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5587d9dfd5e0 con 0x5587d9e9c980 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb37fe640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 0x7f2fa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46620/0 (socket says 172.21.15.129:46620) 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/1598643778 learned_addr learned my addr 172.21.15.129:0/1598643778 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fbad9d640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 0x7f2fa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9e9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.340+0000 7f2fb37fe640 1 -- 172.21.15.129:0/2187559752 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fb40cf540 con 0x7f2fbc106f50 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fba59c640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ea6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/1598643778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 msgr2=0x5587d9ea6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ea6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/1598643778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 msgr2=0x5587d9e9c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9e9c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/1598643778 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d9d99d40 con 0x5587d9ea7240 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fba59c640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ea6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fbad9d640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 0x7f2fa80a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400efd0 tx=0x7f2fb4086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06a100 tx=0x7f2fa4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9d3053012dd150 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.363 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb0ff9640 1 -- 172.21.15.129:0/1598643778 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4035070 con 0x5587d9ea7240 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb0ff9640 1 -- 172.21.15.129:0/1598643778 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa4005020 con 0x5587d9ea7240 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.341+0000 7f2fb0ff9640 1 -- 172.21.15.129:0/1598643778 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40044e0 con 0x5587d9ea7240 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- 172.21.15.129:0/1598643778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 msgr2=0x5587d9ea9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06a100 tx=0x7f2fa4005fd0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- 172.21.15.129:0/1598643778 shutdown_connections 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9ea9630 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ea6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/1598643778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9e9c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- 172.21.15.129:0/1598643778 >> 172.21.15.129:0/1598643778 conn(0x5587d9e985f0 msgr2=0x5587d9e989d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.364 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- 172.21.15.129:0/1598643778 shutdown_connections 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- 172.21.15.129:0/1598643778 wait complete. 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.342+0000 7f2fc2c38740 1 -- start start 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 Processor -- start 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9ea4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 0x5587d9ea4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587d9ea9e10 con 0x5587d9e9c980 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5587d9d89f10 con 0x5587d9e9c060 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5587d9ea9f90 con 0x5587d9ea7240 2024-09-05T01:26:54.365 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fc2c38740 10 client.0 _ll_register_callbacks cb 0x5587d9daddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46628/0 (socket says 172.21.15.129:46628) 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 -- 172.21.15.129:0/2619295661 learned_addr learned my addr 172.21.15.129:0/2619295661 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9ea4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 -- 172.21.15.129:0/2619295661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 msgr2=0x5587d9ea4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 0x5587d9ea4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 0x5587d9ea4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.366 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 -- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 msgr2=0x5587d9ea4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9ea4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 -- 172.21.15.129:0/2619295661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d9b10580 con 0x5587d9e9c980 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 0x5587d9ea4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fba59c640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2fac010a80 tx=0x7f2fac010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9741a2204ef1e597 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.343+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fac01e060 con 0x5587d9e9c980 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fac00fc50 con 0x5587d9e9c980 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa4005020 con 0x5587d9e9c980 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fac025050 con 0x5587d9e9c980 2024-09-05T01:26:54.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 10 client.16515 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5587d9bda960 con 0x5587d9e9c980 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa4005020 con 0x5587d9e9c980 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fac00e3b0 con 0x5587d9e9c980 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 10 client.16515 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 10 client.16515 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 20 client.16515 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9eb6e10 0x5587d9eb9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 4 client.16515 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d9e0eb90 con 0x5587d9eb6e10 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.344+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9eb6e10 0x5587d9eb9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.345+0000 7f2fa1ffb640 10 client.16515 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:54.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.345+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9eb6e10 0x5587d9eb9220 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b9bd2c9f80c706 server_cookie=1213247b85341264 in_seq=0 out_seq=0 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.345+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/2619295661 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587d9e0eb90 con 0x5587d9eb6e10 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.345+0000 7f2fa1ffb640 10 client.16515 handle_command_reply: tid=0 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 client.16515 shutdown 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 msgr2=0x5587d9ea4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2fac010a80 tx=0x7f2fac010f40 comp rx=0 tx=0).stop 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 shutdown_connections 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9eb6e10 0x5587d9eb9220 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9ea7240 0x5587d9ea4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c980 0x5587d9ea4670 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/2619295661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c060 0x5587d9ea4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fa1ffb640 0 client.16515 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:54.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 >> 172.21.15.129:0/2619295661 conn(0x5587d9e985f0 msgr2=0x5587d9e9e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 shutdown_connections 2024-09-05T01:26:54.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 1 -- 172.21.15.129:0/2619295661 wait complete. 2024-09-05T01:26:54.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.346+0000 7f2fc2c38740 20 client.16515 trim_cache size 0 max 16384 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 Processor -- start 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 -- start start 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ebeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9ea7240 0x5587d9ebf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1b16b0 con 0x5587d9e9c060 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1aef80 con 0x5587d9ea7240 2024-09-05T01:26:54.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1acdd0 con 0x5587d9e9c980 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46634/0 (socket says 172.21.15.129:46634) 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/3312061857 learned_addr learned my addr 172.21.15.129:0/3312061857 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/3312061857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 msgr2=0x5587d9ebeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ebeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/3312061857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9ea7240 msgr2=0x5587d9ebf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9ea7240 0x5587d9ebf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.363+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/3312061857 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d9ea30e0 con 0x5587d9e9c060 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x5587d9e99970 tx=0x7f2fa4032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02e5cb686dbe858 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fa3fff640 1 -- 172.21.15.129:0/3312061857 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4035040 con 0x5587d9e9c060 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fa3fff640 1 -- 172.21.15.129:0/3312061857 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa40314a0 con 0x5587d9e9c060 2024-09-05T01:26:54.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fa3fff640 1 -- 172.21.15.129:0/3312061857 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004770 con 0x5587d9e9c060 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 -- 172.21.15.129:0/3312061857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 msgr2=0x5587d9ea60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x5587d9e99970 tx=0x7f2fa4032bb0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 -- 172.21.15.129:0/3312061857 shutdown_connections 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9ea7240 0x5587d9ebf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c980 0x5587d9ebeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/3312061857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9e9c060 0x5587d9ea60a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 -- 172.21.15.129:0/3312061857 >> 172.21.15.129:0/3312061857 conn(0x5587d9e985f0 msgr2=0x5587d9ea5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 -- 172.21.15.129:0/3312061857 shutdown_connections 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.364+0000 7f2fc2c38740 1 -- 172.21.15.129:0/3312061857 wait complete. 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 -- start start 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 Processor -- start 2024-09-05T01:26:54.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c060 0x5587d9eaf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 0x5587d9eaf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1ad440 con 0x5587d9ea7240 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1ae8b0 con 0x5587d9e9c980 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1adb10 con 0x5587d9e9c060 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fc2c38740 10 client.0 _ll_register_callbacks cb 0x7f2fac010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46636/0 (socket says 172.21.15.129:46636) 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/594222739 learned_addr learned my addr 172.21.15.129:0/594222739 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c060 0x5587d9eaf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fba59c640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 0x5587d9eaf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/594222739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c060 msgr2=0x5587d9eaf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c060 0x5587d9eaf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 msgr2=0x5587d9eaf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 0x5587d9eaf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fb9d9b640 1 -- 172.21.15.129:0/594222739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d9ea2000 con 0x5587d9ea7240 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.365+0000 7f2fba59c640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 0x5587d9eaf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fb9d9b640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4032e20 tx=0x7f2fa4005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caeb07e8ed21bea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4035040 con 0x5587d9ea7240 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fa4005490 con 0x5587d9ea7240 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa40314a0 con 0x5587d9ea7240 2024-09-05T01:26:54.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2fa4005690 con 0x5587d9ea7240 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 10 client.16521 fetch_fsmap learned FSMap version 34 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5587d9f21ff0 con 0x5587d9ea7240 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fa40314a0 con 0x5587d9ea7240 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2fa40036f0 con 0x5587d9ea7240 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 10 client.16521 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 10 client.16521 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 20 client.16521 populate_metadata read hostname 'smithi129' 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9f22270 0x5587d9f24660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 4 client.16521 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fdf01480", "release": true, "await": true}] 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.366+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fdf01480", "release": true, "await": true}) -- 0x5587d9e090a0 con 0x5587d9f22270 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.367+0000 7f2fba59c640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9f22270 0x5587d9f24660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:26:54.379 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.367+0000 7f2fa1ffb640 10 client.16521 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:54.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.367+0000 7f2fba59c640 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9f22270 0x5587d9f24660 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a998e63144005851 server_cookie=9109883acd79f25f in_seq=0 out_seq=0 2024-09-05T01:26:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:54 smithi183 ceph-mon[34589]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-09-05T01:26:54.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fa1ffb640 1 -- 172.21.15.129:0/594222739 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5587d9e090a0 con 0x5587d9f22270 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fa1ffb640 10 client.16521 handle_command_reply: tid=0 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 client.16521 shutdown 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 msgr2=0x5587d9eac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4032e20 tx=0x7f2fa4005330 comp rx=0 tx=0).stop 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 shutdown_connections 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5587d9f22270 0x5587d9f24660 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587d9ea7240 0x5587d9eac1b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.425+0000 7f2fa1ffb640 0 client.16521 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5587d9e9c980 0x5587d9eaf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 1 --2- 172.21.15.129:0/594222739 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5587d9e9c060 0x5587d9eaf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 >> 172.21.15.129:0/594222739 conn(0x5587d9e985f0 msgr2=0x5587d9e97750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 shutdown_connections 2024-09-05T01:26:54.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 1 -- 172.21.15.129:0/594222739 wait complete. 2024-09-05T01:26:54.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.426+0000 7f2fc2c38740 20 client.16521 trim_cache size 0 max 16384 2024-09-05T01:26:54.428 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:26:54.428 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 40, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "fdf01480": { 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 40, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:26:54.429 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:54.430 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:26:54.431 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:26:54.431 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 msgr2=0x7f2fa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 0x7f2fa80a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400efd0 tx=0x7f2fb4086040 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 msgr2=0x7f2fbc19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400eaa0 tx=0x7f2fb400eea0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 shutdown_connections 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fa80a2b80 0x7f2fa80a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fbc106f50 0x7f2fbc19cc20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fbc100ca0 0x7f2fbc198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 --2- 172.21.15.129:0/2187559752 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fbc1002f0 0x7f2fbc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 >> 172.21.15.129:0/2187559752 conn(0x7f2fbc077210 msgr2=0x7f2fbc1051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:26:54.432 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 shutdown_connections 2024-09-05T01:26:54.433 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:26:54.427+0000 7f2fc085f640 1 -- 172.21.15.129:0/2187559752 wait complete. 2024-09-05T01:26:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:54 smithi194 ceph-mon[34941]: pgmap v701: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 8.0 MiB/s wr, 581 op/s 2024-09-05T01:26:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 590.0 s, 106.3 tps, lat 1435.549 ms stddev 2749.805 2024-09-05T01:26:55.217 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fdf01480', total seconds elapsed: 13.3 2024-09-05T01:26:56.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:56 smithi183 ceph-mon[34589]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 396 op/s 2024-09-05T01:26:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:56 smithi129 ceph-mon[30979]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 396 op/s 2024-09-05T01:26:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:56 smithi194 ceph-mon[34941]: pgmap v702: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 5.3 MiB/s wr, 396 op/s 2024-09-05T01:26:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:26:58 smithi183 ceph-mon[34589]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.3 MiB/s wr, 458 op/s 2024-09-05T01:26:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:26:58 smithi129 ceph-mon[30979]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.3 MiB/s wr, 458 op/s 2024-09-05T01:26:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:26:58 smithi194 ceph-mon[34941]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.3 MiB/s wr, 458 op/s 2024-09-05T01:27:00.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 595.0 s, 742.0 tps, lat 93.458 ms stddev 148.395 2024-09-05T01:27:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:00 smithi183 ceph-mon[34589]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-09-05T01:27:00.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:00 smithi129 ceph-mon[30979]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-09-05T01:27:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:00 smithi194 ceph-mon[34941]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.7 MiB/s wr, 380 op/s 2024-09-05T01:27:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 -- 172.21.15.129:0/71828075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec095040 msgr2=0x7f2dec0998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 --2- 172.21.15.129:0/71828075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec095040 0x7f2dec0998e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2df4050280 tx=0x7f2df4096bb0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 -- 172.21.15.129:0/71828075 shutdown_connections 2024-09-05T01:27:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 --2- 172.21.15.129:0/71828075 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec095040 0x7f2dec0998e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 --2- 172.21.15.129:0/71828075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec094680 0x7f2dec094b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 --2- 172.21.15.129:0/71828075 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec093920 0x7f2dec093d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 -- 172.21.15.129:0/71828075 >> 172.21.15.129:0/71828075 conn(0x7f2dec08f420 msgr2=0x7f2dec091880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 -- 172.21.15.129:0/71828075 shutdown_connections 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.524+0000 7f2df9c58640 1 -- 172.21.15.129:0/71828075 wait complete. 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 Processor -- start 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 -- start start 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec093920 0x7f2dec131dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec094680 0x7f2dec132300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.529 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec0a7730 con 0x7f2dec093920 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec0a75b0 con 0x7f2dec095040 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9c58640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec0a78b0 con 0x7f2dec094680 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9457640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9457640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:45544/0 (socket says 172.21.15.129:45544) 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df9457640 1 -- 172.21.15.129:0/422991559 learned_addr learned my addr 172.21.15.129:0/422991559 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df3fff640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec094680 0x7f2dec132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.525+0000 7f2df8c56640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec093920 0x7f2dec131dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 -- 172.21.15.129:0/422991559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec094680 msgr2=0x7f2dec132300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec094680 0x7f2dec132300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 -- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec093920 msgr2=0x7f2dec131dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec093920 0x7f2dec131dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 -- 172.21.15.129:0/422991559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dec1399e0 con 0x7f2dec095040 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9457640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2df4067f00 tx=0x7f2df40986f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df1ffb640 1 -- 172.21.15.129:0/422991559 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df40af070 con 0x7f2dec095040 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dec139c70 con 0x7f2dec095040 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df1ffb640 1 -- 172.21.15.129:0/422991559 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df4098d40 con 0x7f2dec095040 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.526+0000 7f2df1ffb640 1 -- 172.21.15.129:0/422991559 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4062890 con 0x7f2dec095040 2024-09-05T01:27:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.527+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dec13a190 con 0x7f2dec095040 2024-09-05T01:27:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.529+0000 7f2df1ffb640 1 -- 172.21.15.129:0/422991559 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2df40b2070 con 0x7f2dec095040 2024-09-05T01:27:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 Processor -- start 2024-09-05T01:27:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 -- start start 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f90f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564694eef2f0 con 0x564694f8bb70 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564694e97270 con 0x564694f8c4b0 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564694eeb2b0 con 0x564694f91540 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46666/0 (socket says 172.21.15.129:46666) 2024-09-05T01:27:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2df3fff640 1 -- 172.21.15.129:0/3300021009 learned_addr learned my addr 172.21.15.129:0/3300021009 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2df1ffb640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 0x7f2de40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.530+0000 7f2df8c56640 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f90f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 -- 172.21.15.129:0/3300021009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 msgr2=0x564694f93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 -- 172.21.15.129:0/3300021009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 msgr2=0x564694f90f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f90f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 -- 172.21.15.129:0/3300021009 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564694e96340 con 0x564694f8bb70 2024-09-05T01:27:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df1ffb640 1 -- 172.21.15.129:0/422991559 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2df415b370 con 0x7f2dec095040 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df9457640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 0x7f2de40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2de00137e0 tx=0x7f2de0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4388ce8687835f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2ddeffd640 1 -- 172.21.15.129:0/3300021009 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0013130 con 0x564694f8bb70 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2ddeffd640 1 -- 172.21.15.129:0/3300021009 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2de00054d0 con 0x564694f8bb70 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.531+0000 7f2ddeffd640 1 -- 172.21.15.129:0/3300021009 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004da0 con 0x564694f8bb70 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.532+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3300021009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 msgr2=0x564694f8bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.532+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2de00137e0 tx=0x7f2de0009610 comp rx=0 tx=0).stop 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.532+0000 7f2df9457640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 0x7f2de40a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2df4068380 tx=0x7f2df4096bb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.539 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.532+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3300021009 shutdown_connections 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.532+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f90f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3300021009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8bb70 0x564694f8bf70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3300021009 >> 172.21.15.129:0/3300021009 conn(0x564694f871e0 msgr2=0x564694f875e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3300021009 shutdown_connections 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3300021009 wait complete. 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 -- start start 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.533+0000 7f2dfdc57740 1 Processor -- start 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f95d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f91540 0x564694f9d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564694f944b0 con 0x564694f91540 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2df8c56640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2df8c56640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:45550/0 (socket says 172.21.15.129:45550) 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2df8c56640 1 -- 172.21.15.129:0/3149470292 learned_addr learned my addr 172.21.15.129:0/3149470292 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564694e78290 con 0x564694f8c4b0 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f95d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x564694f94630 con 0x564694f8bb70 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f91540 0x564694f9d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.534+0000 7f2dfdc57740 10 client.0 _ll_register_callbacks cb 0x564694ea9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 -- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 msgr2=0x564694f962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 -- 172.21.15.129:0/3149470292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f91540 msgr2=0x564694f9d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f91540 0x564694f9d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564694c02210 con 0x564694f8bb70 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df8c56640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2df3fff640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f95d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2de00137e0 tx=0x7f2de0005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340af178a4be848 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:27:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004b20 con 0x564694f8bb70 2024-09-05T01:27:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2de0004cc0 con 0x564694f8bb70 2024-09-05T01:27:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.535+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de00054d0 con 0x564694f8bb70 2024-09-05T01:27:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2de000f7c0 con 0x564694f8bb70 2024-09-05T01:27:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 10 client.25999 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564694cce620 con 0x564694f8bb70 2024-09-05T01:27:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2de00054d0 con 0x564694f8bb70 2024-09-05T01:27:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2de0021990 con 0x564694f8bb70 2024-09-05T01:27:00.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 10 client.25999 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 10 client.25999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 20 client.25999 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564694fa1750 0x564694fa3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 4 client.25999 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.536+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564694efca20 con 0x564694fa1750 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.537+0000 7f2df8c56640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564694fa1750 0x564694fa3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.537+0000 7f2df8c56640 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564694fa1750 0x564694fa3ba0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519f8d613d791d02 server_cookie=5ae340a30fc57bf1 in_seq=0 out_seq=0 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.537+0000 7f2dd3fff640 10 client.25999 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dd3fff640 1 -- 172.21.15.129:0/3149470292 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564694efca20 con 0x564694fa1750 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dd3fff640 10 client.25999 handle_command_reply: tid=0 2024-09-05T01:27:00.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 client.25999 shutdown 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 msgr2=0x564694f95d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f95d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2de00137e0 tx=0x7f2de0005850 comp rx=0 tx=0).stop 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 shutdown_connections 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x564694fa1750 0x564694fa3ba0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f91540 0x564694f9d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8c4b0 0x564694f962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/3149470292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f95d60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 >> 172.21.15.129:0/3149470292 conn(0x564694f871e0 msgr2=0x564694f889f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.538+0000 7f2dd3fff640 0 client.25999 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.539+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 shutdown_connections 2024-09-05T01:27:00.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.539+0000 7f2dfdc57740 1 -- 172.21.15.129:0/3149470292 wait complete. 2024-09-05T01:27:00.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.539+0000 7f2dfdc57740 20 client.25999 trim_cache size 0 max 16384 2024-09-05T01:27:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.563+0000 7f2dfdc57740 1 Processor -- start 2024-09-05T01:27:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.563+0000 7f2dfdc57740 1 -- start start 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f95b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1470a0 con 0x564694f8c4b0 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec144970 con 0x564694f8bb70 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2dfdc57740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1427c0 con 0x564694f91540 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:45560/0 (socket says 172.21.15.129:45560) 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 -- 172.21.15.129:0/995421848 learned_addr learned my addr 172.21.15.129:0/995421848 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df8c56640 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f95b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 -- 172.21.15.129:0/995421848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 msgr2=0x564694f95b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f95b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 -- 172.21.15.129:0/995421848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 msgr2=0x564694f9d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f9d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 -- 172.21.15.129:0/995421848 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564694f958a0 con 0x564694f8bb70 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.564+0000 7f2df3fff640 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2de000f790 tx=0x7f2de00105a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c010dee5a6f33457 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.565+0000 7f2dde7fc640 1 -- 172.21.15.129:0/995421848 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0009510 con 0x564694f8bb70 2024-09-05T01:27:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.565+0000 7f2dde7fc640 1 -- 172.21.15.129:0/995421848 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2de0004640 con 0x564694f8bb70 2024-09-05T01:27:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.565+0000 7f2dde7fc640 1 -- 172.21.15.129:0/995421848 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de0004c60 con 0x564694f8bb70 2024-09-05T01:27:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 -- 172.21.15.129:0/995421848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 msgr2=0x564694f93de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2de000f790 tx=0x7f2de00105a0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 -- 172.21.15.129:0/995421848 shutdown_connections 2024-09-05T01:27:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f91540 0x564694f95b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f9d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/995421848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f8bb70 0x564694f93de0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 -- 172.21.15.129:0/995421848 >> 172.21.15.129:0/995421848 conn(0x564694f871e0 msgr2=0x564694f90ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 -- 172.21.15.129:0/995421848 shutdown_connections 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.566+0000 7f2dfdc57740 1 -- 172.21.15.129:0/995421848 wait complete. 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.567+0000 7f2dfdc57740 1 -- start start 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.567+0000 7f2dfdc57740 1 Processor -- start 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f91540 0x564694f9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:45054/0 (socket says 172.21.15.129:45054) 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46706/0 (socket says 172.21.15.129:46706) 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df3fff640 1 -- 172.21.15.129:0/1812337818 learned_addr learned my addr 172.21.15.129:0/1812337818 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec142e30 con 0x564694f8c4b0 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec1442a0 con 0x564694f91540 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2dec143500 con 0x564694f8bb70 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2dfdc57740 10 client.0 _ll_register_callbacks cb 0x7f2de8009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df3fff640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f91540 0x564694f9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 -- 172.21.15.129:0/1812337818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 msgr2=0x564694f99550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 -- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f91540 msgr2=0x564694f9cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f91540 0x564694f9cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.568+0000 7f2df8c56640 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564694f9d0c0 con 0x564694f8c4b0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2df3fff640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2df8c56640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x564694f8cbc0 tx=0x7f2de8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40983e889437448 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de802c060 con 0x564694f8c4b0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2de8004460 con 0x564694f8c4b0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.569+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de0004640 con 0x564694f8c4b0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.570+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2de8033050 con 0x564694f8c4b0 2024-09-05T01:27:00.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.570+0000 7f2dfdc57740 10 client.16539 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.570+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56469500aff0 con 0x564694f8c4b0 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.570+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2de0004640 con 0x564694f8c4b0 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.570+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2de80047a0 con 0x564694f8c4b0 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 10 client.16539 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 20 client.16539 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56469500b900 0x56469500dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 4 client.16539 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:27:00.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564694ef6d70 con 0x56469500b900 2024-09-05T01:27:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.571+0000 7f2df3fff640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56469500b900 0x56469500dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.572+0000 7f2df3fff640 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56469500b900 0x56469500dd20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19baee1e17e28ab7 server_cookie=d4b083f44e8facd5 in_seq=0 out_seq=0 2024-09-05T01:27:00.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.572+0000 7f2dd3fff640 10 client.16539 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.613+0000 7f2dd3fff640 1 -- 172.21.15.129:0/1812337818 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564694ef6d70 con 0x56469500b900 2024-09-05T01:27:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.613+0000 7f2dd3fff640 10 client.16539 handle_command_reply: tid=0 2024-09-05T01:27:00.614 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.614+0000 7f2dfdc57740 1 client.16539 shutdown 2024-09-05T01:27:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.616+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 msgr2=0x564694f99a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.617 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.616+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x564694f8cbc0 tx=0x7f2de8012b60 comp rx=0 tx=0).stop 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 shutdown_connections 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56469500b900 0x56469500dd20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564694f91540 0x564694f9cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dd3fff640 0 client.16539 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564694f8c4b0 0x564694f99a90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 --2- 172.21.15.129:0/1812337818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x564694f8bb70 0x564694f99550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.619 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.618+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 >> 172.21.15.129:0/1812337818 conn(0x564694f871e0 msgr2=0x564694f95b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.620+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 shutdown_connections 2024-09-05T01:27:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.621+0000 7f2dfdc57740 1 -- 172.21.15.129:0/1812337818 wait complete. 2024-09-05T01:27:00.622 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.621+0000 7f2dfdc57740 20 client.16539 trim_cache size 0 max 16384 2024-09-05T01:27:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:27:00.624 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:27:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:27:00.625 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:27:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.625+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 msgr2=0x7f2de40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.625+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 0x7f2de40a5050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2df4068380 tx=0x7f2df4096bb0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.625+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 msgr2=0x7f2dec139370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.625+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2df4067f00 tx=0x7f2df40986f0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 shutdown_connections 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2de40a2b90 0x7f2de40a5050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dec095040 0x7f2dec139370 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2dec094680 0x7f2dec132300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 --2- 172.21.15.129:0/422991559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2dec093920 0x7f2dec131dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.626+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 >> 172.21.15.129:0/422991559 conn(0x7f2dec08f420 msgr2=0x7f2dec0958a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.627+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 shutdown_connections 2024-09-05T01:27:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:00.627+0000 7f2df9c58640 1 -- 172.21.15.129:0/422991559 wait complete. 2024-09-05T01:27:02.380 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:02 smithi129 ceph-mon[30979]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-09-05T01:27:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:02 smithi183 ceph-mon[34589]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-09-05T01:27:02.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:02 smithi194 ceph-mon[34941]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.4 MiB/s wr, 472 op/s 2024-09-05T01:27:04.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:04 smithi183 ceph-mon[34589]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-09-05T01:27:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:04 smithi129 ceph-mon[30979]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-09-05T01:27:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:04 smithi194 ceph-mon[34941]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.9 MiB/s wr, 652 op/s 2024-09-05T01:27:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 600.0 s, 626.2 tps, lat 214.467 ms stddev 828.448 2024-09-05T01:27:06.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:06 smithi183 ceph-mon[34589]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-05T01:27:06.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:06 smithi129 ceph-mon[30979]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-05T01:27:06.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:06 smithi194 ceph-mon[34941]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 10 MiB/s wr, 724 op/s 2024-09-05T01:27:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:08 smithi183 ceph-mon[34589]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-09-05T01:27:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:08 smithi129 ceph-mon[30979]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-09-05T01:27:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:08 smithi194 ceph-mon[34941]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 13 MiB/s wr, 931 op/s 2024-09-05T01:27:10.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 605.0 s, 739.2 tps, lat 134.976 ms stddev 190.739 2024-09-05T01:27:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:10 smithi183 ceph-mon[34589]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 965 op/s 2024-09-05T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:10 smithi129 ceph-mon[30979]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 965 op/s 2024-09-05T01:27:10.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:10 smithi194 ceph-mon[34941]: pgmap v709: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 14 MiB/s wr, 965 op/s 2024-09-05T01:27:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:12 smithi183 ceph-mon[34589]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 998 op/s 2024-09-05T01:27:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:12 smithi129 ceph-mon[30979]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 998 op/s 2024-09-05T01:27:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:12 smithi194 ceph-mon[34941]: pgmap v710: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 15 MiB/s wr, 998 op/s 2024-09-05T01:27:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94c5ffb640 1 -- 172.21.15.129:0/3963215468 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0030d20 con 0x7f94c8072d20 2024-09-05T01:27:13.784 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 -- 172.21.15.129:0/3963215468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 msgr2=0x7f94c8073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3963215468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c8073180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009be0 tx=0x7f94c002a710 comp rx=0 tx=0).stop 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 -- 172.21.15.129:0/3963215468 shutdown_connections 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3963215468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c8119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3963215468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c8073180 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3963215468 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 0x7f94c8109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 -- 172.21.15.129:0/3963215468 >> 172.21.15.129:0/3963215468 conn(0x7f94c806ece0 msgr2=0x7f94c8071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 -- 172.21.15.129:0/3963215468 shutdown_connections 2024-09-05T01:27:13.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.783+0000 7f94cd76d640 1 -- 172.21.15.129:0/3963215468 wait complete. 2024-09-05T01:27:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.784+0000 7f94cd76d640 1 Processor -- start 2024-09-05T01:27:13.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.784+0000 7f94cd76d640 1 -- start start 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c81a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 0x7f94c81ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c5d0 con 0x7f94c8072d20 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c450 con 0x7f94c80736c0 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94cd76d640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94c811c750 con 0x7f94c8109700 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53614/0 (socket says 172.21.15.129:53614) 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 -- 172.21.15.129:0/3344138061 learned_addr learned my addr 172.21.15.129:0/3344138061 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c81a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c77fe640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 0x7f94c81ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 -- 172.21.15.129:0/3344138061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 msgr2=0x7f94c81ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 0x7f94c81ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 -- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 msgr2=0x7f94c81a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.785+0000 7f94c67fc640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c81a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.786+0000 7f94c67fc640 1 -- 172.21.15.129:0/3344138061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81ae7c0 con 0x7f94c80736c0 2024-09-05T01:27:13.790 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.786+0000 7f94c67fc640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009d00 tx=0x7f94c0030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.790 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T01:27:13.791 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:27:13.792 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.788+0000 7f94affff640 1 -- 172.21.15.129:0/3344138061 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0041020 con 0x7f94c80736c0 2024-09-05T01:27:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.788+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81aea50 con 0x7f94c80736c0 2024-09-05T01:27:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.788+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c81aef40 con 0x7f94c80736c0 2024-09-05T01:27:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.788+0000 7f94affff640 1 -- 172.21.15.129:0/3344138061 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94c0037040 con 0x7f94c80736c0 2024-09-05T01:27:13.793 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.788+0000 7f94affff640 1 -- 172.21.15.129:0/3344138061 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c0036070 con 0x7f94c80736c0 2024-09-05T01:27:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.793+0000 7f94affff640 1 -- 172.21.15.129:0/3344138061 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f94c0044020 con 0x7f94c80736c0 2024-09-05T01:27:13.794 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.793+0000 7f94d05e6740 1 Processor -- start 2024-09-05T01:27:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.793+0000 7f94d05e6740 1 -- start start 2024-09-05T01:27:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664f1370 0x55b8664f3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b86644f2f0 con 0x55b8664eb9a0 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8663f7270 con 0x55b8664ec2e0 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94d05e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b86644b2b0 con 0x55b8664f1370 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49824/0 (socket says 172.21.15.129:49824) 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.794+0000 7f94c77fe640 1 -- 172.21.15.129:0/4008888913 learned_addr learned my addr 172.21.15.129:0/4008888913 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.795+0000 7f94affff640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 0x7f94b00a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.795+0000 7f94affff640 1 -- 172.21.15.129:0/3344138061 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94c00eff40 con 0x7f94c80736c0 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.795+0000 7f94c77fe640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 0x7f94b00a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.801 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.795+0000 7f94c67fc640 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664f1370 0x55b8664f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 -- 172.21.15.129:0/4008888913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664f1370 msgr2=0x55b8664f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664f1370 0x55b8664f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 -- 172.21.15.129:0/4008888913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 msgr2=0x55b8664f0da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664f0da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 -- 172.21.15.129:0/4008888913 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8663f6340 con 0x55b8664eb9a0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94b8013ac0 tx=0x7f94b8013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cb3d86a33d4c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.796+0000 7f94c77fe640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 0x7f94b00a5330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f94b800f850 tx=0x7f94b801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.797+0000 7f94ad7fa640 1 -- 172.21.15.129:0/4008888913 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8023070 con 0x55b8664eb9a0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.797+0000 7f94ad7fa640 1 -- 172.21.15.129:0/4008888913 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b8009e50 con 0x55b8664eb9a0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.798+0000 7f94ad7fa640 1 -- 172.21.15.129:0/4008888913 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b8013050 con 0x55b8664eb9a0 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.798+0000 7f94d05e6740 1 -- 172.21.15.129:0/4008888913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 msgr2=0x55b8664ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.798+0000 7f94d05e6740 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f94b8013ac0 tx=0x7f94b8013f80 comp rx=0 tx=0).stop 2024-09-05T01:27:13.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 -- 172.21.15.129:0/4008888913 shutdown_connections 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664f1370 0x55b8664f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 --2- 172.21.15.129:0/4008888913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664ebda0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 -- 172.21.15.129:0/4008888913 >> 172.21.15.129:0/4008888913 conn(0x55b8664e7010 msgr2=0x55b8664e7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 -- 172.21.15.129:0/4008888913 shutdown_connections 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 -- 172.21.15.129:0/4008888913 wait complete. 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 -- start start 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.799+0000 7f94d05e6740 1 Processor -- start 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664ec2e0 0x55b8664f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664fd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b8664f42e0 con 0x55b8664ec2e0 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b8663d8290 con 0x55b8664f1370 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b8664f4460 con 0x55b8664eb9a0 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94d05e6740 10 client.0 _ll_register_callbacks cb 0x55b866409140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94c6ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94c6ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48782/0 (socket says 172.21.15.129:48782) 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94c6ffd640 1 -- 172.21.15.129:0/992619080 learned_addr learned my addr 172.21.15.129:0/992619080 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:13.805 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94c77fe640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.800+0000 7f94c67fc640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664ec2e0 0x55b8664f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 -- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 msgr2=0x55b8664fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 -- 172.21.15.129:0/992619080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664ec2e0 msgr2=0x55b8664f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664ec2e0 0x55b8664f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 -- 172.21.15.129:0/992619080 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b866162210 con 0x55b8664eb9a0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00e450 tx=0x7f94bc00e910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba1a6fe4d2e95348 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc017070 con 0x55b8664eb9a0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94bc013680 con 0x55b8664eb9a0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.801+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b8009e50 con 0x55b8664eb9a0 2024-09-05T01:27:13.806 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94bc021050 con 0x55b8664eb9a0 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 10 client.26014 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b86622e620 con 0x55b8664eb9a0 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b8009e50 con 0x55b8664eb9a0 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f94bc01ca20 con 0x55b8664eb9a0 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 10 client.26014 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 10 client.26014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 20 client.26014 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b866501a10 0x55b866503e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 4 client.26014 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:13.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.802+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b86645ca20 con 0x55b866501a10 2024-09-05T01:27:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.806+0000 7f94c67fc640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b866501a10 0x55b866503e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.808+0000 7f94c67fc640 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b866501a10 0x55b866503e50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd6d9b8518728c0 server_cookie=aa0c41c9ed381e02 in_seq=0 out_seq=0 2024-09-05T01:27:13.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.808+0000 7f94aaff5640 10 client.26014 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.808+0000 7f94aaff5640 1 -- 172.21.15.129:0/992619080 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b86645ca20 con 0x55b866501a10 2024-09-05T01:27:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.808+0000 7f94aaff5640 10 client.26014 handle_command_reply: tid=0 2024-09-05T01:27:13.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.808+0000 7f94d05e6740 1 client.26014 shutdown 2024-09-05T01:27:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.809+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 msgr2=0x55b8664f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.809+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00e450 tx=0x7f94bc00e910 comp rx=0 tx=0).stop 2024-09-05T01:27:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 shutdown_connections 2024-09-05T01:27:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b866501a10 0x55b866503e50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664ec2e0 0x55b8664f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 --2- 172.21.15.129:0/992619080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664f5c80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 >> 172.21.15.129:0/992619080 conn(0x55b8664e7010 msgr2=0x55b8664e8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.810+0000 7f94aaff5640 0 client.26014 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.813+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 shutdown_connections 2024-09-05T01:27:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.814+0000 7f94d05e6740 1 -- 172.21.15.129:0/992619080 wait complete. 2024-09-05T01:27:13.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.815+0000 7f94d05e6740 20 client.26014 trim_cache size 0 max 16384 2024-09-05T01:27:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 Processor -- start 2024-09-05T01:27:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 -- start start 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664e8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664ec2e0 0x55b8664fd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94b40127b0 con 0x55b8664eb9a0 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94b4010080 con 0x55b8664f1370 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.844+0000 7f94d05e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94b400ded0 con 0x55b8664ec2e0 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53640/0 (socket says 172.21.15.129:53640) 2024-09-05T01:27:13.846 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 -- 172.21.15.129:0/2013197791 learned_addr learned my addr 172.21.15.129:0/2013197791 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664e8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 -- 172.21.15.129:0/2013197791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664ec2e0 msgr2=0x55b8664fd190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:27:13.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664ec2e0 0x55b8664fd190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 -- 172.21.15.129:0/2013197791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 msgr2=0x55b8664e8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664e8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.845+0000 7f94c77fe640 1 -- 172.21.15.129:0/2013197791 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8664f57c0 con 0x55b8664f1370 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.846+0000 7f94c77fe640 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94b80094b0 tx=0x7f94b8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ffe22124b1a86f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.846+0000 7f94acff9640 1 -- 172.21.15.129:0/2013197791 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b802c070 con 0x55b8664f1370 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.846+0000 7f94acff9640 1 -- 172.21.15.129:0/2013197791 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b8009c00 con 0x55b8664f1370 2024-09-05T01:27:13.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 -- 172.21.15.129:0/2013197791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 msgr2=0x55b8664e8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f94b80094b0 tx=0x7f94b8009600 comp rx=0 tx=0).stop 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 -- 172.21.15.129:0/2013197791 shutdown_connections 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664f1370 0x55b8664e8940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664ec2e0 0x55b8664fd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2013197791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664eb9a0 0x55b8664e8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.847+0000 7f94d05e6740 1 -- 172.21.15.129:0/2013197791 >> 172.21.15.129:0/2013197791 conn(0x55b8664e7010 msgr2=0x55b8664e61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.848+0000 7f94d05e6740 1 -- 172.21.15.129:0/2013197791 shutdown_connections 2024-09-05T01:27:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.848+0000 7f94d05e6740 1 -- 172.21.15.129:0/2013197791 wait complete. 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.848+0000 7f94d05e6740 1 -- start start 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.848+0000 7f94d05e6740 1 Processor -- start 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664f1370 0x55b8664fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94b400e540 con 0x55b8664f1370 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94b400f9b0 con 0x55b8664ec2e0 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94b400ec10 con 0x55b8664eb9a0 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94d05e6740 10 client.0 _ll_register_callbacks cb 0x7f94bc009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c67fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48794/0 (socket says 172.21.15.129:48794) 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c67fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53648/0 (socket says 172.21.15.129:53648) 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 -- 172.21.15.129:0/2671616112 learned_addr learned my addr 172.21.15.129:0/2671616112 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c77fe640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664f1370 0x55b8664fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 -- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 msgr2=0x55b8664fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 -- 172.21.15.129:0/2671616112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664f1370 msgr2=0x55b8664fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664f1370 0x55b8664fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c6ffd640 1 -- 172.21.15.129:0/2671616112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8664fc8c0 con 0x55b8664eb9a0 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.849+0000 7f94c67fc640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94c6ffd640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006fd0 tx=0x7f94bc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac2997d30cb999c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc02a020 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b8009c00 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94bc01c640 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94bc035050 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94d05e6740 10 client.26017 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.850+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b86657dca0 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94b8009c00 con 0x55b8664eb9a0 2024-09-05T01:27:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f94bc0038c0 con 0x55b8664eb9a0 2024-09-05T01:27:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 10 client.26017 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 10 client.26017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 20 client.26017 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b86657e650 0x55b866580a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 4 client.26017 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:27:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b866456d70 con 0x55b86657e650 2024-09-05T01:27:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.851+0000 7f94c67fc640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b86657e650 0x55b866580a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.852+0000 7f94aaff5640 10 client.26017 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.852+0000 7f94c67fc640 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b86657e650 0x55b866580a70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a740ca6d00530342 server_cookie=6b59641d46137dbd in_seq=0 out_seq=0 2024-09-05T01:27:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.925+0000 7f94aaff5640 1 -- 172.21.15.129:0/2671616112 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b866456d70 con 0x55b86657e650 2024-09-05T01:27:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.925+0000 7f94aaff5640 10 client.26017 handle_command_reply: tid=0 2024-09-05T01:27:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.926+0000 7f94d05e6740 1 client.26017 shutdown 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.926+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 msgr2=0x55b8664fadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.926+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f94bc006fd0 tx=0x7f94bc004270 comp rx=0 tx=0).stop 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 shutdown_connections 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b86657e650 0x55b866580a70 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8664f1370 0x55b8664fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b8664ec2e0 0x55b8664fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 --2- 172.21.15.129:0/2671616112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b8664eb9a0 0x55b8664fadb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94aaff5640 0 client.26017 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 >> 172.21.15.129:0/2671616112 conn(0x55b8664e7010 msgr2=0x55b8664e73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 shutdown_connections 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 1 -- 172.21.15.129:0/2671616112 wait complete. 2024-09-05T01:27:13.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.927+0000 7f94d05e6740 20 client.26017 trim_cache size 0 max 16384 2024-09-05T01:27:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:27:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:27:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:27:13.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:27:13.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 msgr2=0x7f94b00a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 0x7f94b00a5330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f94b800f850 tx=0x7f94b801e000 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 msgr2=0x7f94c81a70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f94c0009d00 tx=0x7f94c0030fc0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 shutdown_connections 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94b00a2e70 0x7f94b00a5330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94c8109700 0x7f94c81ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94c80736c0 0x7f94c81a70e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 --2- 172.21.15.129:0/3344138061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94c8072d20 0x7f94c81a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.929+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 >> 172.21.15.129:0/3344138061 conn(0x7f94c806ece0 msgr2=0x7f94c80716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.930+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 shutdown_connections 2024-09-05T01:27:13.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:13.930+0000 7f94cd76d640 1 -- 172.21.15.129:0/3344138061 wait complete. 2024-09-05T01:27:14.286 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:14 smithi183 ceph-mon[34589]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 992 op/s 2024-09-05T01:27:14.437 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:14 smithi129 ceph-mon[30979]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 992 op/s 2024-09-05T01:27:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:14 smithi194 ceph-mon[34941]: pgmap v711: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 992 op/s 2024-09-05T01:27:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 -- 172.21.15.129:0/2527141109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428112d20 msgr2=0x7f9428115110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 --2- 172.21.15.129:0/2527141109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428112d20 0x7f9428115110 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9418009900 tx=0x7f941802a4a0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 -- 172.21.15.129:0/2527141109 shutdown_connections 2024-09-05T01:27:14.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 --2- 172.21.15.129:0/2527141109 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9428112d20 0x7f9428115110 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 --2- 172.21.15.129:0/2527141109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f94281126b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 --2- 172.21.15.129:0/2527141109 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94281098a0 0x7f9428109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 -- 172.21.15.129:0/2527141109 >> 172.21.15.129:0/2527141109 conn(0x7f942806e140 msgr2=0x7f942806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 -- 172.21.15.129:0/2527141109 shutdown_connections 2024-09-05T01:27:14.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.934+0000 7f942e155640 1 -- 172.21.15.129:0/2527141109 wait complete. 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 Processor -- start 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 -- start start 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94281098a0 0x7f94281121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428112d20 0x7f942810c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9428117140 con 0x7f942810a1c0 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9428116fc0 con 0x7f94281098a0 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.935+0000 7f942e155640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94281172c0 con 0x7f9428112d20 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49858/0 (socket says 172.21.15.129:49858) 2024-09-05T01:27:14.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 -- 172.21.15.129:0/2834214725 learned_addr learned my addr 172.21.15.129:0/2834214725 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9427fff640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428112d20 0x7f942810c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f94277fe640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94281098a0 0x7f94281121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 -- 172.21.15.129:0/2834214725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428112d20 msgr2=0x7f942810c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428112d20 0x7f942810c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 -- 172.21.15.129:0/2834214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94281098a0 msgr2=0x7f94281121a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94281098a0 0x7f94281121a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.936+0000 7f9426ffd640 1 -- 172.21.15.129:0/2834214725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942810cd20 con 0x7f942810a1c0 2024-09-05T01:27:14.938 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.937+0000 7f9426ffd640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f942000ea50 tx=0x7f942000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.939+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000cd60 con 0x7f942810a1c0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.939+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9420011920 con 0x7f942810a1c0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.939+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942001a9a0 con 0x7f942810a1c0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.939+0000 7f942e155640 1 -- 172.21.15.129:0/2834214725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942810d010 con 0x7f942810a1c0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.940+0000 7f942e155640 1 -- 172.21.15.129:0/2834214725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281bc3d0 con 0x7f942810a1c0 2024-09-05T01:27:14.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.941+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9428073b00 con 0x7f942810a1c0 2024-09-05T01:27:14.947 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.946+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9420011470 con 0x7f942810a1c0 2024-09-05T01:27:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.947+0000 7f9424ff9640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 0x7f94100a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:14.948 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.947+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94200cfb20 con 0x7f942810a1c0 2024-09-05T01:27:14.949 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.948+0000 7f94277fe640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 0x7f94100a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:14.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.949+0000 7f94277fe640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 0x7f94100a5180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f941c009bd0 tx=0x7f941c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:14.953 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:14.952+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9420016030 con 0x7f942810a1c0 2024-09-05T01:27:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 610.0 s, 767.0 tps, lat 130.242 ms stddev 72.299 2024-09-05T01:27:15.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.343+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f942810d750 con 0x7f942810a1c0 2024-09-05T01:27:15.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.348+0000 7f9424ff9640 1 -- 172.21.15.129:0/2834214725 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f942009c720 con 0x7f942810a1c0 2024-09-05T01:27:15.349 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:27:15.350 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:27:15.350 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:27:15.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 msgr2=0x7f94100a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:15.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 0x7f94100a5180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f941c009bd0 tx=0x7f941c005f90 comp rx=0 tx=0).stop 2024-09-05T01:27:15.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 msgr2=0x7f942810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:15.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f942000ea50 tx=0x7f942000ef10 comp rx=0 tx=0).stop 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 shutdown_connections 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f94100a2cc0 0x7f94100a5180 secure :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f941c009bd0 tx=0x7f941c005f90 comp rx=0 tx=0).stop 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9428112d20 0x7f942810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f942810a1c0 0x7f942810c220 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.354+0000 7f940e7fc640 1 --2- 172.21.15.129:0/2834214725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94281098a0 0x7f94281121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.355+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 >> 172.21.15.129:0/2834214725 conn(0x7f942806e140 msgr2=0x7f9428070940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.355+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 shutdown_connections 2024-09-05T01:27:15.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:15.355+0000 7f940e7fc640 1 -- 172.21.15.129:0/2834214725 wait complete. 2024-09-05T01:27:15.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:15 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2834214725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:15.705 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:15 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2834214725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:15.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:15 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2834214725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:16.076 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T01:27:16.077 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T01:27:16.597 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:16.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:16 smithi183 ceph-mon[34589]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 874 op/s 2024-09-05T01:27:16.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:16 smithi129 ceph-mon[30979]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 874 op/s 2024-09-05T01:27:16.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:16 smithi194 ceph-mon[34941]: pgmap v712: 129 pgs: 129 active+clean; 1.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 13 MiB/s wr, 874 op/s 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1003556951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c410c160 msgr2=0x7f77c410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1003556951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c410c160 0x7f77c410e600 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009ca0 tx=0x7f77bc02a950 comp rx=0 tx=0).stop 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1003556951 shutdown_connections 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1003556951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c410c160 0x7f77c410e600 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1003556951 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c4103650 0x7f77c410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1003556951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4102d30 0x7f77c4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1003556951 >> 172.21.15.129:0/1003556951 conn(0x7f77c40fc790 msgr2=0x7f77c40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1003556951 shutdown_connections 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.179+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1003556951 wait complete. 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 Processor -- start 2024-09-05T01:27:17.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 -- start start 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4103650 0x7f77c4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c410c160 0x7f77c4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4111e10 con 0x7f77c4102d30 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4111c90 con 0x7f77c4103650 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c9ad5640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77c4111f90 con 0x7f77c410c160 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49878/0 (socket says 172.21.15.129:49878) 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.180+0000 7f77c37fe640 1 -- 172.21.15.129:0/1075829766 learned_addr learned my addr 172.21.15.129:0/1075829766 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c3fff640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c410c160 0x7f77c4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4103650 0x7f77c4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 -- 172.21.15.129:0/1075829766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c410c160 msgr2=0x7f77c4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c410c160 0x7f77c4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 -- 172.21.15.129:0/1075829766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4103650 msgr2=0x7f77c4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4103650 0x7f77c4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 -- 172.21.15.129:0/1075829766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c41a6870 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c37fe640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77b400e9f0 tx=0x7f77b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c0ff9640 1 -- 172.21.15.129:0/1075829766 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b400cd80 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c41a6b60 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c0ff9640 1 -- 172.21.15.129:0/1075829766 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77b40108e0 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c41a70f0 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.181+0000 7f77c0ff9640 1 -- 172.21.15.129:0/1075829766 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b4018920 con 0x7f77c4102d30 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 Processor -- start 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 -- start start 2024-09-05T01:27:17.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x56217778e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x562177790d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5621776e92a0 con 0x562177784080 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562177690b10 con 0x56217778e940 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.184+0000 7f77cbeae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5621776e5260 con 0x562177783760 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48838/0 (socket says 172.21.15.129:48838) 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 -- 172.21.15.129:0/3863028583 learned_addr learned my addr 172.21.15.129:0/3863028583 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x562177790d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 -- 172.21.15.129:0/3863028583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 msgr2=0x562177790d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x562177790d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 -- 172.21.15.129:0/3863028583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 msgr2=0x56217778e400 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x56217778e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 -- 172.21.15.129:0/3863028583 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217745ab90 con 0x562177783760 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c0ff9640 1 -- 172.21.15.129:0/1075829766 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f77b4010a90 con 0x7f77c4102d30 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77c3fff640 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009a20 tx=0x7f77bc009360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89c68964ebb4bf10 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77a9ffb640 1 -- 172.21.15.129:0/3863028583 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc034040 con 0x562177783760 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77a9ffb640 1 -- 172.21.15.129:0/3863028583 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77bc02fec0 con 0x562177783760 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.185+0000 7f77a9ffb640 1 -- 172.21.15.129:0/3863028583 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0042d0 con 0x562177783760 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.186+0000 7f77c0ff9640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.186+0000 7f77c0ff9640 1 -- 172.21.15.129:0/1075829766 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f77b40cf830 con 0x7f77c4102d30 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.186+0000 7f77c37fe640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- 172.21.15.129:0/3863028583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 msgr2=0x562177783b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009a20 tx=0x7f77bc009360 comp rx=0 tx=0).stop 2024-09-05T01:27:17.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77c37fe640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009380 tx=0x7f77b4097f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- 172.21.15.129:0/3863028583 shutdown_connections 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x562177790d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x56217778e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 --2- 172.21.15.129:0/3863028583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177783760 0x562177783b40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- 172.21.15.129:0/3863028583 >> 172.21.15.129:0/3863028583 conn(0x56217777fcf0 msgr2=0x5621777800d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- 172.21.15.129:0/3863028583 shutdown_connections 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- 172.21.15.129:0/3863028583 wait complete. 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 -- start start 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.187+0000 7f77cbeae740 1 Processor -- start 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177784080 0x56217778bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x56217778c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562177791510 con 0x562177783760 2024-09-05T01:27:17.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5621776716e0 con 0x562177784080 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562177791690 con 0x56217778e940 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77cbeae740 10 client.0 _ll_register_callbacks cb 0x562177695a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49890/0 (socket says 172.21.15.129:49890) 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 -- 172.21.15.129:0/1229876183 learned_addr learned my addr 172.21.15.129:0/1229876183 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x56217778c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177784080 0x56217778bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 -- 172.21.15.129:0/1229876183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 msgr2=0x56217778c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x56217778c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 -- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177784080 msgr2=0x56217778bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177784080 0x56217778bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 -- 172.21.15.129:0/1229876183 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217779aa60 con 0x562177783760 2024-09-05T01:27:17.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77c3fff640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009620 tx=0x7f77bc02f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c8f4ffcd78979b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.188+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc034070 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc02fec0 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f77bc003e30 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77bc0040b0 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 10 client.16572 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562177791510 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77bc02fec0 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f77bc04f040 con 0x562177783760 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 10 client.16572 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 10 client.16572 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 20 client.16572 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:17.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217779e550 0x5621777a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 4 client.16572 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621776f69d0 con 0x56217779e550 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.189+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217779e550 0x5621777a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.190+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217779e550 0x5621777a0940 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7829a7240f2ae9 server_cookie=1ac9e8ed109040f in_seq=0 out_seq=0 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.190+0000 7f77a2ffd640 10 client.16572 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.190+0000 7f77a2ffd640 1 -- 172.21.15.129:0/1229876183 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621776f69d0 con 0x56217779e550 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.190+0000 7f77a2ffd640 10 client.16572 handle_command_reply: tid=0 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.190+0000 7f77cbeae740 1 client.16572 shutdown 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 msgr2=0x56217778b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009620 tx=0x7f77bc02f7d0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 shutdown_connections 2024-09-05T01:27:17.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217779e550 0x5621777a0940 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x56217778c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177784080 0x56217778bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 --2- 172.21.15.129:0/1229876183 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x56217778b840 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77a2ffd640 0 client.16572 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 >> 172.21.15.129:0/1229876183 conn(0x56217777fcf0 msgr2=0x562177785a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 shutdown_connections 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 1 -- 172.21.15.129:0/1229876183 wait complete. 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.191+0000 7f77cbeae740 20 client.16572 trim_cache size 0 max 16384 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.207+0000 7f77cbeae740 1 Processor -- start 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.207+0000 7f77cbeae740 1 -- start start 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177784080 0x562177785e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.535 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x5621777a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41af600 con 0x562177783760 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41aced0 con 0x56217778e940 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77cbeae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41aad40 con 0x562177784080 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49892/0 (socket says 172.21.15.129:49892) 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 -- 172.21.15.129:0/4256485936 learned_addr learned my addr 172.21.15.129:0/4256485936 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x5621777a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177784080 0x562177785e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 -- 172.21.15.129:0/4256485936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177784080 msgr2=0x562177785e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177784080 0x562177785e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 -- 172.21.15.129:0/4256485936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 msgr2=0x5621777a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.536 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x5621777a6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 -- 172.21.15.129:0/4256485936 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217777fb20 con 0x562177783760 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77c3fff640 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009430 tx=0x7f77bc02f560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1a696e7996b4f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77a97fa640 1 -- 172.21.15.129:0/4256485936 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc034070 con 0x562177783760 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77a97fa640 1 -- 172.21.15.129:0/4256485936 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77bc03f070 con 0x562177783760 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.208+0000 7f77a97fa640 1 -- 172.21.15.129:0/4256485936 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc030440 con 0x562177783760 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- 172.21.15.129:0/4256485936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 msgr2=0x5621777858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009430 tx=0x7f77bc02f560 comp rx=0 tx=0).stop 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- 172.21.15.129:0/4256485936 shutdown_connections 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56217778e940 0x5621777a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562177784080 0x562177785e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 --2- 172.21.15.129:0/4256485936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177783760 0x5621777858c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- 172.21.15.129:0/4256485936 >> 172.21.15.129:0/4256485936 conn(0x56217777fcf0 msgr2=0x5621777851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.537 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- 172.21.15.129:0/4256485936 shutdown_connections 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- 172.21.15.129:0/4256485936 wait complete. 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.209+0000 7f77cbeae740 1 -- start start 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 Processor -- start 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 0x562177796e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x562177793e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ab390 con 0x562177784080 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41ac800 con 0x562177783760 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f77c41aba60 con 0x56217778e940 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49902/0 (socket says 172.21.15.129:49902) 2024-09-05T01:27:17.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c2ffd640 1 -- 172.21.15.129:0/477656 learned_addr learned my addr 172.21.15.129:0/477656 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77cbeae740 10 client.0 _ll_register_callbacks cb 0x7f77bc02fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c3fff640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x562177793e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c3fff640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 0x562177796e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c2ffd640 1 -- 172.21.15.129:0/477656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 msgr2=0x562177793e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.210+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x562177793e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77c2ffd640 1 -- 172.21.15.129:0/477656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 msgr2=0x562177796e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 0x562177796e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77c2ffd640 1 -- 172.21.15.129:0/477656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562177780830 con 0x562177784080 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77c3fff640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 0x562177796e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77c2ffd640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x56217778b0d0 tx=0x7f77b801bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b8da4e5f27ab50 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8010880 con 0x562177784080 2024-09-05T01:27:17.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f77b8010a20 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77bc03f070 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f77b8034050 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 10 client.16578 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562177809be0 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77bc03f070 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f77b802f030 con 0x562177784080 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 10 client.16578 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 10 client.16578 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 20 client.16578 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217780a450 0x56217780c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 4 client.16578 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.211+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5621776f0d20 con 0x56217780a450 2024-09-05T01:27:17.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.212+0000 7f77c3fff640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217780a450 0x56217780c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:17.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.212+0000 7f77a2ffd640 10 client.16578 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:17.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.212+0000 7f77c3fff640 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217780a450 0x56217780c870 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d780728805c020da server_cookie=df575928c1e3366f in_seq=0 out_seq=0 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77a2ffd640 1 -- 172.21.15.129:0/477656 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5621776f0d20 con 0x56217780a450 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77a2ffd640 10 client.16578 handle_command_reply: tid=0 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 client.16578 shutdown 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 msgr2=0x5621777938e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x56217778b0d0 tx=0x7f77b801bce0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 shutdown_connections 2024-09-05T01:27:17.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56217780a450 0x56217780c870 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56217778e940 0x562177793e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562177784080 0x5621777938e0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77a2ffd640 0 client.16578 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 --2- 172.21.15.129:0/477656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562177783760 0x562177796e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 >> 172.21.15.129:0/477656 conn(0x56217777fcf0 msgr2=0x5621777859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 shutdown_connections 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 1 -- 172.21.15.129:0/477656 wait complete. 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.554+0000 7f77cbeae740 20 client.16578 trim_cache size 0 max 16384 2024-09-05T01:27:17.556 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 42, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "392081af": { 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 42, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:27:17.557 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:17.558 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 msgr2=0x7f77ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009380 tx=0x7f77b4097f70 comp rx=0 tx=0).stop 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 msgr2=0x7f77c4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f77b400e9f0 tx=0x7f77b400eeb0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 shutdown_connections 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f77ac0a2c00 0x7f77ac0a50c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f77c410c160 0x7f77c4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f77c4103650 0x7f77c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 --2- 172.21.15.129:0/1075829766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77c4102d30 0x7f77c4100990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:17.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 >> 172.21.15.129:0/1075829766 conn(0x7f77c40fc790 msgr2=0x7f77c40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:17.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 shutdown_connections 2024-09-05T01:27:17.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:17.555+0000 7f77c9ad5640 1 -- 172.21.15.129:0/1075829766 wait complete. 2024-09-05T01:27:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:18 smithi129 ceph-mon[30979]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-09-05T01:27:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:18 smithi183 ceph-mon[34589]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-09-05T01:27:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:18 smithi194 ceph-mon[34941]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 14 MiB/s wr, 997 op/s 2024-09-05T01:27:18.699 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '392081af', quiesce took 4.9 seconds. Will release after: 5.1 2024-09-05T01:27:19.886 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:19 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:27:19.910 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:19 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:27:19.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:19 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:27:20.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 615.0 s, 251.9 tps, lat 154.601 ms stddev 64.757 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:27:21.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:27:21.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:27:21.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:20 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:27:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:27:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:27:21.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:20 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:27:21.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:20 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:27:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:22 smithi183 ceph-mon[34589]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-09-05T01:27:22.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:22 smithi129 ceph-mon[30979]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-09-05T01:27:22.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:22 smithi194 ceph-mon[34941]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.2 MiB/s wr, 723 op/s 2024-09-05T01:27:23.801 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:27:24.060 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:24 smithi183 ceph-mon[34589]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-09-05T01:27:24.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:24 smithi129 ceph-mon[30979]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-09-05T01:27:24.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:24 smithi194 ceph-mon[34941]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-09-05T01:27:25.212 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 620.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:27:25.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 -- 172.21.15.129:0/824384540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9868102d40 msgr2=0x7f9868103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:25.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 --2- 172.21.15.129:0/824384540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9868102d40 0x7f9868103120 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9858009900 tx=0x7f985802a480 comp rx=0 tx=0).stop 2024-09-05T01:27:25.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 -- 172.21.15.129:0/824384540 shutdown_connections 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 --2- 172.21.15.129:0/824384540 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f986810c170 0x7f986810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 --2- 172.21.15.129:0/824384540 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 0x7f986810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 --2- 172.21.15.129:0/824384540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9868102d40 0x7f9868103120 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 -- 172.21.15.129:0/824384540 >> 172.21.15.129:0/824384540 conn(0x7f98680fc7a0 msgr2=0x7f98680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 -- 172.21.15.129:0/824384540 shutdown_connections 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.480+0000 7f986ce08640 1 -- 172.21.15.129:0/824384540 wait complete. 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 Processor -- start 2024-09-05T01:27:25.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 -- start start 2024-09-05T01:27:25.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 0x7f9868198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 0x7f9868198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9868111e20 con 0x7f986810c170 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9868111ca0 con 0x7f9868102d40 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986ce08640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9868111fa0 con 0x7f9868103660 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51424/0 (socket says 172.21.15.129:51424) 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 -- 172.21.15.129:0/313955200 learned_addr learned my addr 172.21.15.129:0/313955200 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9865d9b640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 0x7f9868198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986659c640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 0x7f9868198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 -- 172.21.15.129:0/313955200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 msgr2=0x7f9868198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:25.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 0x7f9868198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 -- 172.21.15.129:0/313955200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 msgr2=0x7f9868198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 0x7f9868198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f9866d9d640 1 -- 172.21.15.129:0/313955200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986819d2f0 con 0x7f986810c170 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.481+0000 7f986659c640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 0x7f9868198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f9866d9d640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f986000eaa0 tx=0x7f986000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000cdb0 con 0x7f986810c170 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9860005020 con 0x7f986810c170 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986819d5e0 con 0x7f986810c170 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600116d0 con 0x7f986810c170 2024-09-05T01:27:25.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.482+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a4ec0 con 0x7f986810c170 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.483+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9868103120 con 0x7f986810c170 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.484+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f98600051d0 con 0x7f986810c170 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.484+0000 7f98577fe640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 0x7f98500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.485+0000 7f986659c640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 0x7f98500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.485+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f98600cf2c0 con 0x7f986810c170 2024-09-05T01:27:25.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.485+0000 7f986659c640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 0x7f98500a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9858000c00 tx=0x7f9858031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:25.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.489+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9860016030 con 0x7f986810c170 2024-09-05T01:27:25.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.689+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9868106c50 con 0x7f986810c170 2024-09-05T01:27:25.690 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.690+0000 7f98577fe640 1 -- 172.21.15.129:0/313955200 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f986009be10 con 0x7f986810c170 2024-09-05T01:27:25.691 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:27:25.691 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:27:25.691 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 msgr2=0x7f98500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 0x7f98500a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9858000c00 tx=0x7f9858031400 comp rx=0 tx=0).stop 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 msgr2=0x7f986819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f986000eaa0 tx=0x7f986000ef60 comp rx=0 tx=0).stop 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 shutdown_connections 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f98500a2c00 0x7f98500a50c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f986810c170 0x7f986819cc10 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9868103660 0x7f9868198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 --2- 172.21.15.129:0/313955200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9868102d40 0x7f9868198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:25.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 >> 172.21.15.129:0/313955200 conn(0x7f98680fc7a0 msgr2=0x7f98680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:25.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 shutdown_connections 2024-09-05T01:27:25.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:25.692+0000 7f986ce08640 1 -- 172.21.15.129:0/313955200 wait complete. 2024-09-05T01:27:25.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/313955200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:26.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/313955200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:26.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/313955200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:26.335 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 392081af --release --await' 2024-09-05T01:27:26.335 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 392081af --release --await 2024-09-05T01:27:26.585 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:26.973 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:26 smithi129 ceph-mon[30979]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2024-09-05T01:27:27.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:26 smithi194 ceph-mon[34941]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- 172.21.15.129:0/76164613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44100a50 msgr2=0x7f8c44100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- 172.21.15.129:0/76164613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44100a50 0x7f8c44100e50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009900 tx=0x7f8c4002a410 comp rx=0 tx=0).stop 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- 172.21.15.129:0/76164613 shutdown_connections 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- 172.21.15.129:0/76164613 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 0x7f8c44108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- 172.21.15.129:0/76164613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44101c40 0x7f8c441020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- 172.21.15.129:0/76164613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44100a50 0x7f8c44100e50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- 172.21.15.129:0/76164613 >> 172.21.15.129:0/76164613 conn(0x7f8c440fc1b0 msgr2=0x7f8c440fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- 172.21.15.129:0/76164613 shutdown_connections 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- 172.21.15.129:0/76164613 wait complete. 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 Processor -- start 2024-09-05T01:27:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- start start 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44100a50 0x7f8c44197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 0x7f8c4419f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4410c560 con 0x7f8c44101c40 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4410c3e0 con 0x7f8c44100a50 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.035+0000 7f8c4b959640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4410c6e0 con 0x7f8c44102600 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51440/0 (socket says 172.21.15.129:51440) 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 -- 172.21.15.129:0/2555762592 learned_addr learned my addr 172.21.15.129:0/2555762592 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 0x7f8c4419f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44100a50 0x7f8c44197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 -- 172.21.15.129:0/2555762592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 msgr2=0x7f8c4419f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.039 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 0x7f8c4419f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 -- 172.21.15.129:0/2555762592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44100a50 msgr2=0x7f8c44197c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44100a50 0x7f8c44197c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 -- 172.21.15.129:0/2555762592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4419f850 con 0x7f8c44101c40 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c920 tx=0x7f8c3400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c3a7fc640 1 -- 172.21.15.129:0/2555762592 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34007c70 con 0x7f8c44101c40 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4419fb40 con 0x7f8c44101c40 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c3a7fc640 1 -- 172.21.15.129:0/2555762592 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c34007e10 con 0x7f8c44101c40 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c441a00e0 con 0x7f8c44101c40 2024-09-05T01:27:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.036+0000 7f8c3a7fc640 1 -- 172.21.15.129:0/2555762592 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34018820 con 0x7f8c44101c40 2024-09-05T01:27:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 Processor -- start 2024-09-05T01:27:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 -- start start 2024-09-05T01:27:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 0x556da21aada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21ab370 0x556da21ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556da21092f0 con 0x556da21ab370 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556da20b1270 con 0x556da21a62e0 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556da21052b0 con 0x556da21a59a0 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 0x556da21aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c49ef6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 0x556da21aada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:54262/0 (socket says 172.21.15.129:54262) 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.039+0000 7f8c496f5640 1 -- 172.21.15.129:0/1109154266 learned_addr learned my addr 172.21.15.129:0/1109154266 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 -- 172.21.15.129:0/1109154266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 msgr2=0x556da21aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 0x556da21aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 -- 172.21.15.129:0/1109154266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21ab370 msgr2=0x556da21ad830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21ab370 0x556da21ad830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 -- 172.21.15.129:0/1109154266 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da20b0340 con 0x556da21a59a0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c3a7fc640 1 -- 172.21.15.129:0/2555762592 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8c340189c0 con 0x7f8c44101c40 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a5da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007e40 tx=0x7f8c3c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d489ba10ea09e66a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c2f7fe640 1 -- 172.21.15.129:0/1109154266 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c003b70 con 0x556da21a59a0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c2f7fe640 1 -- 172.21.15.129:0/1109154266 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c3c003d10 con 0x556da21a59a0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c2f7fe640 1 -- 172.21.15.129:0/1109154266 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004490 con 0x556da21a59a0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.040+0000 7f8c3a7fc640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 0x7f8c300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 0x7f8c300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c3a7fc640 1 -- 172.21.15.129:0/2555762592 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8c340cf560 con 0x7f8c44101c40 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/1109154266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 msgr2=0x556da21a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a5da0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c007e40 tx=0x7f8c3c00f790 comp rx=0 tx=0).stop 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/1109154266 shutdown_connections 2024-09-05T01:27:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21ab370 0x556da21ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21a62e0 0x556da21aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/1109154266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a5da0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/1109154266 >> 172.21.15.129:0/1109154266 conn(0x556da21a1010 msgr2=0x556da21a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 0x7f8c300a50f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c400096c0 tx=0x7f8c40000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/1109154266 shutdown_connections 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/1109154266 wait complete. 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 -- start start 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.041+0000 7f8c4e7d0740 1 Processor -- start 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556da21ae340 con 0x556da21a59a0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556da2092290 con 0x556da21ab370 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556da21ae4c0 con 0x556da21a62e0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c4e7d0740 10 client.0 _ll_register_callbacks cb 0x556da20c3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51446/0 (socket says 172.21.15.129:51446) 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 -- 172.21.15.129:0/2651644543 learned_addr learned my addr 172.21.15.129:0/2651644543 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 -- 172.21.15.129:0/2651644543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 msgr2=0x556da21b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 -- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 msgr2=0x556da21b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.042+0000 7f8c496f5640 1 -- 172.21.15.129:0/2651644543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da1e1c210 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c496f5640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8c40033000 tx=0x7f8c4002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b1fd0b81f063d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40035040 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8c400037f0 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c3c003d10 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c40003a30 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 10 client.16593 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556da1ee8620 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c3c003d10 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8c40044030 con 0x556da21a59a0 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 10 client.16593 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 20 client.16593 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da21bb870 0x556da21bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 4 client.16593 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.043+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556da2116a20 con 0x556da21bb870 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.044+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da21bb870 0x556da21bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.044+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da21bb870 0x556da21bdc80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b19b5407d2ec398 server_cookie=69997f58128a8c00 in_seq=0 out_seq=0 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.044+0000 7f8c2cff9640 10 client.16593 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c2cff9640 1 -- 172.21.15.129:0/2651644543 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556da2116a20 con 0x556da21bb870 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c2cff9640 10 client.16593 handle_command_reply: tid=0 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 client.16593 shutdown 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 msgr2=0x556da21afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8c40033000 tx=0x7f8c4002a410 comp rx=0 tx=0).stop 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 shutdown_connections 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da21bb870 0x556da21bdc80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c2cff9640 0 client.16593 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/2651644543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21afc20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 >> 172.21.15.129:0/2651644543 conn(0x556da21a1010 msgr2=0x556da21a2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 shutdown_connections 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/2651644543 wait complete. 2024-09-05T01:27:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.045+0000 7f8c4e7d0740 20 client.16593 trim_cache size 0 max 16384 2024-09-05T01:27:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 Processor -- start 2024-09-05T01:27:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 -- start start 2024-09-05T01:27:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.062 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441acfc0 con 0x556da21a62e0 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441aa890 con 0x556da21ab370 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441a86e0 con 0x556da21a59a0 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51460/0 (socket says 172.21.15.129:51460) 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 -- 172.21.15.129:0/4136347307 learned_addr learned my addr 172.21.15.129:0/4136347307 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c496f5640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 -- 172.21.15.129:0/4136347307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 msgr2=0x556da21a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 -- 172.21.15.129:0/4136347307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 msgr2=0x556da21a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 -- 172.21.15.129:0/4136347307 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556da21a0e30 con 0x556da21a62e0 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c496f5640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.060+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400fda0 tx=0x7f8c34086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e407ed9bfc9d68 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c2effd640 1 -- 172.21.15.129:0/4136347307 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3409b9b0 con 0x556da21a62e0 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c2effd640 1 -- 172.21.15.129:0/4136347307 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c340cf040 con 0x556da21a62e0 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c2effd640 1 -- 172.21.15.129:0/4136347307 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34098120 con 0x556da21a62e0 2024-09-05T01:27:27.064 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/4136347307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 msgr2=0x556da21b7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400fda0 tx=0x7f8c34086040 comp rx=0 tx=0).stop 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/4136347307 shutdown_connections 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a62e0 0x556da21b7110 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/4136347307 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a59a0 0x556da21a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/4136347307 >> 172.21.15.129:0/4136347307 conn(0x556da21a1010 msgr2=0x556da21af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/4136347307 shutdown_connections 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.061+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/4136347307 wait complete. 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 -- start start 2024-09-05T01:27:27.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 Processor -- start 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441a8d50 con 0x556da21a59a0 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441aa1c0 con 0x556da21ab370 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8c441a9420 con 0x556da21a62e0 2024-09-05T01:27:27.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c4e7d0740 10 client.0 _ll_register_callbacks cb 0x7f8c40032d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c496f5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51476/0 (socket says 172.21.15.129:51476) 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c496f5640 1 -- 172.21.15.129:0/3728347830 learned_addr learned my addr 172.21.15.129:0/3728347830 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c49ef6640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c496f5640 1 -- 172.21.15.129:0/3728347830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 msgr2=0x556da21b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.062+0000 7f8c496f5640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c496f5640 1 -- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 msgr2=0x556da21b5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c496f5640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c496f5640 1 -- 172.21.15.129:0/3728347830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556da21b6540 con 0x556da21a59a0 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c496f5640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006010 tx=0x7f8c40005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ac0e570f1571b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c40035040 con 0x556da21a59a0 2024-09-05T01:27:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8c40003dd0 con 0x556da21a59a0 2024-09-05T01:27:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c340cf040 con 0x556da21a59a0 2024-09-05T01:27:27.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8c40067050 con 0x556da21a59a0 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c4e7d0740 10 client.16599 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556da22267b0 con 0x556da21a59a0 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.063+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c340cf040 con 0x556da21a59a0 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8c40044070 con 0x556da21a59a0 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 10 client.16599 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 20 client.16599 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da2227160 0x556da2229580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 4 client.16599 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556da21ab0f0 con 0x556da2227160 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da2227160 0x556da2229580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c48ef4640 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da2227160 0x556da2229580 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451d2198629dcb6d server_cookie=ae29e942ca7d7635 in_seq=0 out_seq=0 2024-09-05T01:27:27.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.064+0000 7f8c2cff9640 10 client.16599 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c2cff9640 1 -- 172.21.15.129:0/3728347830 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556da21ab0f0 con 0x556da2227160 2024-09-05T01:27:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c2cff9640 10 client.16599 handle_command_reply: tid=0 2024-09-05T01:27:27.083 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 client.16599 shutdown 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 msgr2=0x556da21b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006010 tx=0x7f8c40005850 comp rx=0 tx=0).stop 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 shutdown_connections 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556da2227160 0x556da2229580 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556da21ab370 0x556da21b5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c2cff9640 0 client.16599 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556da21a62e0 0x556da21b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 --2- 172.21.15.129:0/3728347830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556da21a59a0 0x556da21b4a30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 >> 172.21.15.129:0/3728347830 conn(0x556da21a1010 msgr2=0x556da219f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 shutdown_connections 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.082+0000 7f8c4e7d0740 1 -- 172.21.15.129:0/3728347830 wait complete. 2024-09-05T01:27:27.084 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4e7d0740 20 client.16599 trim_cache size 0 max 16384 2024-09-05T01:27:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:27:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:27:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:27:27.085 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 msgr2=0x7f8c300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 0x7f8c300a50f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c400096c0 tx=0x7f8c40000bc0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 msgr2=0x7f8c44198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400c920 tx=0x7f8c3400cde0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 shutdown_connections 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8c300a2c30 0x7f8c300a50f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8c44102600 0x7f8c4419f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c44101c40 0x7f8c44198180 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 --2- 172.21.15.129:0/2555762592 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8c44100a50 0x7f8c44197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.086 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 >> 172.21.15.129:0/2555762592 conn(0x7f8c440fc1b0 msgr2=0x7f8c440fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 shutdown_connections 2024-09-05T01:27:27.087 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:27.083+0000 7f8c4b959640 1 -- 172.21.15.129:0/2555762592 wait complete. 2024-09-05T01:27:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 -- 172.21.15.129:0/2458070441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 msgr2=0x7f1cec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/2458070441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec10da40 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009ca0 tx=0x7f1ce402a950 comp rx=0 tx=0).stop 2024-09-05T01:27:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 -- 172.21.15.129:0/2458070441 shutdown_connections 2024-09-05T01:27:27.107 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/2458070441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec10da40 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/2458070441 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec102570 0x7f1cec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/2458070441 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec100e20 0x7f1cec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 -- 172.21.15.129:0/2458070441 >> 172.21.15.129:0/2458070441 conn(0x7f1cec0fc7d0 msgr2=0x7f1cec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 -- 172.21.15.129:0/2458070441 shutdown_connections 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 -- 172.21.15.129:0/2458070441 wait complete. 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.066+0000 7f1cf24d0640 1 Processor -- start 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 -- start start 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec102570 0x7f1cec199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec110e30 con 0x7f1cec102ef0 2024-09-05T01:27:27.108 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec110cb0 con 0x7f1cec102570 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf24d0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec110fb0 con 0x7f1cec100e20 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf0a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec102570 0x7f1cec199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:60260/0 (socket says 172.21.15.129:60260) 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf0a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51496/0 (socket says 172.21.15.129:51496) 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 -- 172.21.15.129:0/4281737064 learned_addr learned my addr 172.21.15.129:0/4281737064 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 -- 172.21.15.129:0/4281737064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec102570 msgr2=0x7f1cec199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec102570 0x7f1cec199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 -- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 msgr2=0x7f1cec1a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cebfff640 1 -- 172.21.15.129:0/4281737064 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1a47e0 con 0x7f1cec100e20 2024-09-05T01:27:27.109 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.067+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1cebfff640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00ea60 tx=0x7f1cdc00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1ce97fa640 1 -- 172.21.15.129:0/4281737064 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00cde0 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1ce97fa640 1 -- 172.21.15.129:0/4281737064 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cdc0035f0 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1ce97fa640 1 -- 172.21.15.129:0/4281737064 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc010690 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1a49b0 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.068+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1a4e70 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.072+0000 7f1cf48a9740 1 Processor -- start 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.072+0000 7f1ce97fa640 1 -- 172.21.15.129:0/4281737064 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1cdc0037a0 con 0x7f1cec100e20 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.072+0000 7f1cf48a9740 1 -- start start 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538585e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x565385858bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5653857b4620 con 0x565385859260 2024-09-05T01:27:27.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56538575be50 con 0x56538585ebc0 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5653857b05e0 con 0x56538585e2a0 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51498/0 (socket says 172.21.15.129:51498) 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/550747319 learned_addr learned my addr 172.21.15.129:0/550747319 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ce97fa640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 0x7f1cd40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cebfff640 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x565385858bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/550747319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 msgr2=0x56538585e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538585e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/550747319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 msgr2=0x565385858bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x565385858bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/550747319 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538574cd40 con 0x565385859260 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1cebfff640 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x565385858bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.073+0000 7f1ce97fa640 1 -- 172.21.15.129:0/4281737064 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1cdc0cf5c0 con 0x7f1cec100e20 2024-09-05T01:27:27.111 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.074+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009c20 tx=0x7f1ce0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3435cf96a0fa3c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.074+0000 7f1cd27fc640 1 -- 172.21.15.129:0/550747319 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004230 con 0x565385859260 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.074+0000 7f1cd27fc640 1 -- 172.21.15.129:0/550747319 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ce00043d0 con 0x565385859260 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.074+0000 7f1cd27fc640 1 -- 172.21.15.129:0/550747319 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0004c80 con 0x565385859260 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.074+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 0x7f1cd40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 0x7f1cd40a5190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005e30 tx=0x7f1ce402fc70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- 172.21.15.129:0/550747319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 msgr2=0x5653858596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009c20 tx=0x7f1ce0011810 comp rx=0 tx=0).stop 2024-09-05T01:27:27.112 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- 172.21.15.129:0/550747319 shutdown_connections 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x5653858596e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x565385858bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/550747319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538585e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- 172.21.15.129:0/550747319 >> 172.21.15.129:0/550747319 conn(0x56538584b0a0 msgr2=0x56538584b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- 172.21.15.129:0/550747319 shutdown_connections 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- 172.21.15.129:0/550747319 wait complete. 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 -- start start 2024-09-05T01:27:27.113 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.075+0000 7f1cf48a9740 1 Processor -- start 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x565385862080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x5653858625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5653858655e0 con 0x56538585ebc0 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56538573cf10 con 0x565385859260 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x565385865760 con 0x56538585e2a0 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf48a9740 10 client.0 _ll_register_callbacks cb 0x565385760db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x565385862080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.114 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51512/0 (socket says 172.21.15.129:51512) 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 -- 172.21.15.129:0/3956891597 learned_addr learned my addr 172.21.15.129:0/3956891597 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x5653858625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 -- 172.21.15.129:0/3956891597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 msgr2=0x5653858625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x5653858625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 -- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 msgr2=0x565385862080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x565385862080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 -- 172.21.15.129:0/3956891597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653854c3580 con 0x56538585ebc0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4035000 tx=0x7f1ce402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f6b4e579e47f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1ccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.076+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4034040 con 0x56538585ebc0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce00043d0 con 0x56538585ebc0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1ce4004040 con 0x56538585ebc0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ce4004240 con 0x56538585ebc0 2024-09-05T01:27:27.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 10 client.16611 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56538558d960 con 0x56538585ebc0 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ce00043d0 con 0x56538585ebc0 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1ce4050b20 con 0x56538585ebc0 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 10 client.16611 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 10 client.16611 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 20 client.16611 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565385872480 0x5653858748b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 4 client.16611 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.077+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653857c1b90 con 0x565385872480 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.078+0000 7f1cebfff640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565385872480 0x5653858748b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.078+0000 7f1ccb7fe640 10 client.16611 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.078+0000 7f1cebfff640 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565385872480 0x5653858748b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f42e3a823da38e server_cookie=1c6ece3f6b927bac in_seq=0 out_seq=0 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/3956891597 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653857c1b90 con 0x565385872480 2024-09-05T01:27:27.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1ccb7fe640 10 client.16611 handle_command_reply: tid=0 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 client.16611 shutdown 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 msgr2=0x565385862b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4035000 tx=0x7f1ce402a950 comp rx=0 tx=0).stop 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 shutdown_connections 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565385872480 0x5653858748b0 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585ebc0 0x565385862b00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1ccb7fe640 0 client.16611 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x5653858625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3956891597 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x565385862080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.079+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 >> 172.21.15.129:0/3956891597 conn(0x56538584b0a0 msgr2=0x56538584a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.080+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 shutdown_connections 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.080+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3956891597 wait complete. 2024-09-05T01:27:27.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.080+0000 7f1cf48a9740 20 client.16611 trim_cache size 0 max 16384 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 Processor -- start 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 -- start start 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538584a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x56538587a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1af450 con 0x565385859260 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1acd20 con 0x56538585ebc0 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1aab70 con 0x56538585e2a0 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51522/0 (socket says 172.21.15.129:51522) 2024-09-05T01:27:27.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 -- 172.21.15.129:0/3812188511 learned_addr learned my addr 172.21.15.129:0/3812188511 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538584a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x56538587a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 -- 172.21.15.129:0/3812188511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 msgr2=0x56538584a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538584a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 -- 172.21.15.129:0/3812188511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 msgr2=0x56538587a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x56538587a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 -- 172.21.15.129:0/3812188511 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56538585bdc0 con 0x565385859260 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.097+0000 7f1cebfff640 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0cd020 tx=0x7f1cdc004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ae576e33297e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cd1ffb640 1 -- 172.21.15.129:0/3812188511 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0ce040 con 0x565385859260 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cd1ffb640 1 -- 172.21.15.129:0/3812188511 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cdc09ba10 con 0x565385859260 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cd1ffb640 1 -- 172.21.15.129:0/3812188511 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc099b40 con 0x565385859260 2024-09-05T01:27:27.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3812188511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 msgr2=0x565385858bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc0cd020 tx=0x7f1cdc004560 comp rx=0 tx=0).stop 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3812188511 shutdown_connections 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56538585ebc0 0x56538587a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585e2a0 0x56538584a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/3812188511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565385859260 0x565385858bf0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.121 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3812188511 >> 172.21.15.129:0/3812188511 conn(0x56538584b0a0 msgr2=0x565385849f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3812188511 shutdown_connections 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.098+0000 7f1cf48a9740 1 -- 172.21.15.129:0/3812188511 wait complete. 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 -- start start 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 Processor -- start 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x56538586aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 0x565385867f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1ab1e0 con 0x56538585e2a0 2024-09-05T01:27:27.122 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1ac650 con 0x565385859260 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1ab8b0 con 0x56538585ebc0 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cf48a9740 10 client.0 _ll_register_callbacks cb 0x7f1ce402f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51534/0 (socket says 172.21.15.129:51534) 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/1373189670 learned_addr learned my addr 172.21.15.129:0/1373189670 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:27.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.099+0000 7f1cebfff640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x56538586aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 0x565385867f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/1373189670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 msgr2=0x565385867f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 0x565385867f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 msgr2=0x56538586aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x56538586aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 -- 172.21.15.129:0/1373189670 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653858684c0 con 0x56538585e2a0 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cf0a6d640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 0x565385867f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cebfff640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x56538586aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ceb7fe640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x56538584c8f0 tx=0x7f1ce0009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ec48c0bf8a301e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0028070 con 0x56538585e2a0 2024-09-05T01:27:27.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1ce0004680 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cdc09ba10 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ce0034050 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cf48a9740 10 client.16617 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653858ddf10 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.100+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cdc09ba10 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1ce0018070 con 0x56538585e2a0 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 10 client.16617 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 10 client.16617 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:27.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 20 client.16617 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653858de580 0x5653858e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 4 client.16617 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "392081af", "release": true, "await": true}] 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "392081af", "release": true, "await": true}) -- 0x5653857bc0a0 con 0x5653858de580 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cebfff640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653858de580 0x5653858e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1cebfff640 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653858de580 0x5653858e09a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0babc86e476fa9e server_cookie=866b20b42cb5e574 in_seq=0 out_seq=0 2024-09-05T01:27:27.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.101+0000 7f1ccb7fe640 10 client.16617 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1ccb7fe640 1 -- 172.21.15.129:0/1373189670 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5653857bc0a0 con 0x5653858de580 2024-09-05T01:27:27.159 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1ccb7fe640 10 client.16617 handle_command_reply: tid=0 2024-09-05T01:27:27.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 client.16617 shutdown 2024-09-05T01:27:27.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 msgr2=0x5653858679c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x56538584c8f0 tx=0x7f1ce0009eb0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 shutdown_connections 2024-09-05T01:27:27.160 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653858de580 0x5653858e09a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56538585ebc0 0x565385867f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1ccb7fe640 0 client.16617 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56538585e2a0 0x5653858679c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 --2- 172.21.15.129:0/1373189670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x565385859260 0x56538586aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 >> 172.21.15.129:0/1373189670 conn(0x56538584b0a0 msgr2=0x56538584a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 shutdown_connections 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 1 -- 172.21.15.129:0/1373189670 wait complete. 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:27:27.161 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 44, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "392081af": { 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 44, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:27:27.162 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:27.163 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.159+0000 7f1cf48a9740 20 client.16617 trim_cache size 0 max 16384 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 msgr2=0x7f1cd40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 0x7f1cd40a5190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005e30 tx=0x7f1ce402fc70 comp rx=0 tx=0).stop 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 msgr2=0x7f1cec199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00ea60 tx=0x7f1cdc00ef20 comp rx=0 tx=0).stop 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 shutdown_connections 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1cd40a2cd0 0x7f1cd40a5190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cec102ef0 0x7f1cec1a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1cec102570 0x7f1cec199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 --2- 172.21.15.129:0/4281737064 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1cec100e20 0x7f1cec199220 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:27.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 >> 172.21.15.129:0/4281737064 conn(0x7f1cec0fc7d0 msgr2=0x7f1cec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:27.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 shutdown_connections 2024-09-05T01:27:27.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:27.160+0000 7f1cf24d0640 1 -- 172.21.15.129:0/4281737064 wait complete. 2024-09-05T01:27:27.165 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:26 smithi183 ceph-mon[34589]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2024-09-05T01:27:27.937 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '392081af', total seconds elapsed: 14.1 2024-09-05T01:27:28.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:28 smithi129 ceph-mon[30979]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-05T01:27:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:28 smithi183 ceph-mon[34589]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-05T01:27:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:28 smithi194 ceph-mon[34941]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 3.2 MiB/s wr, 332 op/s 2024-09-05T01:27:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 625.0 s, 511.5 tps, lat 510.399 ms stddev 1943.657 2024-09-05T01:27:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:30 smithi183 ceph-mon[34589]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-05T01:27:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:30 smithi129 ceph-mon[30979]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-05T01:27:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:30 smithi194 ceph-mon[34941]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-05T01:27:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:32 smithi183 ceph-mon[34589]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T01:27:32.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:32 smithi129 ceph-mon[30979]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T01:27:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:32 smithi194 ceph-mon[34941]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-09-05T01:27:34.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:34 smithi183 ceph-mon[34589]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.6 MiB/s wr, 392 op/s 2024-09-05T01:27:34.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:34 smithi129 ceph-mon[30979]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.6 MiB/s wr, 392 op/s 2024-09-05T01:27:34.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:34 smithi194 ceph-mon[34941]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.6 MiB/s wr, 392 op/s 2024-09-05T01:27:35.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 630.0 s, 773.8 tps, lat 128.814 ms stddev 96.808 2024-09-05T01:27:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:36 smithi183 ceph-mon[34589]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 471 op/s 2024-09-05T01:27:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:36 smithi129 ceph-mon[30979]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 471 op/s 2024-09-05T01:27:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:36 smithi194 ceph-mon[34941]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.4 MiB/s wr, 471 op/s 2024-09-05T01:27:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:38 smithi183 ceph-mon[34589]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 733 op/s 2024-09-05T01:27:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:38 smithi129 ceph-mon[30979]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 733 op/s 2024-09-05T01:27:38.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:38 smithi194 ceph-mon[34941]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.3 MiB/s wr, 733 op/s 2024-09-05T01:27:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 635.0 s, 612.5 tps, lat 159.851 ms stddev 190.949 2024-09-05T01:27:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 -- 172.21.15.129:0/2928860503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2730101c30 msgr2=0x7f27301020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 --2- 172.21.15.129:0/2928860503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2730101c30 0x7f27301020b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2724009860 tx=0x7f272402a340 comp rx=0 tx=0).stop 2024-09-05T01:27:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 -- 172.21.15.129:0/2928860503 shutdown_connections 2024-09-05T01:27:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 --2- 172.21.15.129:0/2928860503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f27301025f0 0x7f2730108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 --2- 172.21.15.129:0/2928860503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2730101c30 0x7f27301020b0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2724009860 tx=0x7f272402a340 comp rx=0 tx=0).stop 2024-09-05T01:27:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 --2- 172.21.15.129:0/2928860503 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2730100a40 0x7f2730100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 -- 172.21.15.129:0/2928860503 >> 172.21.15.129:0/2928860503 conn(0x7f27300fc1e0 msgr2=0x7f27300fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 -- 172.21.15.129:0/2928860503 shutdown_connections 2024-09-05T01:27:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.224+0000 7f273528d640 1 -- 172.21.15.129:0/2928860503 wait complete. 2024-09-05T01:27:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.225+0000 7f273528d640 1 Processor -- start 2024-09-05T01:27:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.225+0000 7f273528d640 1 -- start start 2024-09-05T01:27:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.225+0000 7f273528d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.225+0000 7f273528d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27301025f0 0x7f273019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.225+0000 7f273528d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27301a3ad0 0x7f27301a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:39372/0 (socket says 172.21.15.129:39372) 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 -- 172.21.15.129:0/1598554249 learned_addr learned my addr 172.21.15.129:0/1598554249 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f273528d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d6c0 con 0x7f2730100a40 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d540 con 0x7f27301025f0 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d840 con 0x7f27301a3ad0 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27301a3ad0 0x7f27301a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 -- 172.21.15.129:0/1598554249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27301a3ad0 msgr2=0x7f27301a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27301a3ad0 0x7f27301a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 -- 172.21.15.129:0/1598554249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27301025f0 msgr2=0x7f273019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27301025f0 0x7f273019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.226+0000 7f272effd640 1 -- 172.21.15.129:0/1598554249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2730192e70 con 0x7f2730100a40 2024-09-05T01:27:40.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.227+0000 7f272effd640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9a0 tx=0x7f272000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.227+0000 7f2717fff640 1 -- 172.21.15.129:0/1598554249 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272000cd80 con 0x7f2730100a40 2024-09-05T01:27:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.227+0000 7f2717fff640 1 -- 172.21.15.129:0/1598554249 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27200108e0 con 0x7f2730100a40 2024-09-05T01:27:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.227+0000 7f2717fff640 1 -- 172.21.15.129:0/1598554249 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720018950 con 0x7f2730100a40 2024-09-05T01:27:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.227+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2730193160 con 0x7f2730100a40 2024-09-05T01:27:40.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.228+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301936b0 con 0x7f2730100a40 2024-09-05T01:27:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 Processor -- start 2024-09-05T01:27:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 -- start start 2024-09-05T01:27:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f2717fff640 1 -- 172.21.15.129:0/1598554249 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2720010a90 con 0x7f2730100a40 2024-09-05T01:27:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209d9a0 0x558b3209dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320a5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b320012f0 con 0x558b3209d9a0 2024-09-05T01:27:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b31fa9270 con 0x558b320a3370 2024-09-05T01:27:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.233+0000 7f273802b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558b31ffd2b0 con 0x558b3209e2e0 2024-09-05T01:27:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f272effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f272effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:58710/0 (socket says 172.21.15.129:58710) 2024-09-05T01:27:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f272effd640 1 -- 172.21.15.129:0/3818883098 learned_addr learned my addr 172.21.15.129:0/3818883098 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f272f7fe640 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209d9a0 0x558b3209dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f272e7fc640 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.234+0000 7f2717fff640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 0x7f27180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.235+0000 7f2717fff640 1 -- 172.21.15.129:0/1598554249 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f27200cf8b0 con 0x7f2730100a40 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.237+0000 7f272effd640 1 -- 172.21.15.129:0/3818883098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 msgr2=0x558b320a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.237+0000 7f272effd640 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.237+0000 7f272effd640 1 -- 172.21.15.129:0/3818883098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209d9a0 msgr2=0x558b3209dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.237+0000 7f272effd640 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209d9a0 0x558b3209dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.237+0000 7f272effd640 1 -- 172.21.15.129:0/3818883098 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31fa8340 con 0x558b3209e2e0 2024-09-05T01:27:40.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.239+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 0x7f27180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.241+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 0x7f27180a5260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2728012710 tx=0x7f2728004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.243+0000 7f272effd640 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f272009b9a0 tx=0x7f2720000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c16cdcf49e90af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.244+0000 7f27157fa640 1 -- 172.21.15.129:0/3818883098 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720010e20 con 0x558b3209e2e0 2024-09-05T01:27:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.244+0000 7f27157fa640 1 -- 172.21.15.129:0/3818883098 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272009c3a0 con 0x558b3209e2e0 2024-09-05T01:27:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.244+0000 7f27157fa640 1 -- 172.21.15.129:0/3818883098 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272009b0f0 con 0x558b3209e2e0 2024-09-05T01:27:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.246+0000 7f273802b740 1 -- 172.21.15.129:0/3818883098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 msgr2=0x558b320a2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.249 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.246+0000 7f273802b740 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f272009b9a0 tx=0x7f2720000bc0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.248+0000 7f273802b740 1 -- 172.21.15.129:0/3818883098 shutdown_connections 2024-09-05T01:27:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.248+0000 7f273802b740 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.250 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.248+0000 7f273802b740 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320a2da0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.248+0000 7f273802b740 1 --2- 172.21.15.129:0/3818883098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209d9a0 0x558b3209dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.248+0000 7f273802b740 1 -- 172.21.15.129:0/3818883098 >> 172.21.15.129:0/3818883098 conn(0x558b32099010 msgr2=0x558b32099410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.249+0000 7f273802b740 1 -- 172.21.15.129:0/3818883098 shutdown_connections 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.249+0000 7f273802b740 1 -- 172.21.15.129:0/3818883098 wait complete. 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.249+0000 7f273802b740 1 -- start start 2024-09-05T01:27:40.251 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.249+0000 7f273802b740 1 Processor -- start 2024-09-05T01:27:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.250+0000 7f273802b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b320a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.250+0000 7f273802b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209e2e0 0x558b320a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.250+0000 7f273802b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f273802b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558b320a62d0 con 0x558b3209e2e0 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f273802b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558b31f8a290 con 0x558b320a3370 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58694/0 (socket says 172.21.15.129:58694) 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 -- 172.21.15.129:0/1336148832 learned_addr learned my addr 172.21.15.129:0/1336148832 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558b320a6450 con 0x558b3209d9a0 2024-09-05T01:27:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f273802b740 10 client.0 _ll_register_callbacks cb 0x558b31fbb140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:40.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272e7fc640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209e2e0 0x558b320a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 -- 172.21.15.129:0/1336148832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 msgr2=0x558b320a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b320a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 -- 172.21.15.129:0/1336148832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209e2e0 msgr2=0x558b320a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.251+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209e2e0 0x558b320a80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.252+0000 7f272f7fe640 1 -- 172.21.15.129:0/1336148832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b31d14210 con 0x558b320a3370 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.252+0000 7f272f7fe640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f272800eed0 tx=0x7f2728004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b87708f3bc003cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.252+0000 7f2712ff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.252+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728024020 con 0x558b320a3370 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.253+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272009c3a0 con 0x558b320a3370 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.253+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2728020610 con 0x558b320a3370 2024-09-05T01:27:40.255 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.253+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f272802f050 con 0x558b320a3370 2024-09-05T01:27:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.256+0000 7f273802b740 10 client.26198 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.256+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b31de0620 con 0x558b320a3370 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2728019040 con 0x558b320a3370 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f272800f770 con 0x558b320a3370 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 10 client.26198 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 20 client.26198 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b320b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 4 client.26198 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.257+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f272009c3a0 con 0x558b320b3cb0 2024-09-05T01:27:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.258+0000 7f272effd640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b320b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.259+0000 7f2712ff5640 10 client.26198 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.260+0000 7f272effd640 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b320b6150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6859e2de294afd5 server_cookie=1f8668c864794572 in_seq=0 out_seq=0 2024-09-05T01:27:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.264+0000 7f2712ff5640 1 -- 172.21.15.129:0/1336148832 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f272009c3a0 con 0x558b320b3cb0 2024-09-05T01:27:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.264+0000 7f2712ff5640 10 client.26198 handle_command_reply: tid=0 2024-09-05T01:27:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.266+0000 7f273802b740 1 client.26198 shutdown 2024-09-05T01:27:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.272+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 msgr2=0x558b320af130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.272+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f272800eed0 tx=0x7f2728004a40 comp rx=0 tx=0).stop 2024-09-05T01:27:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.273+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 shutdown_connections 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.273+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b320b6150 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.273+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b320a3370 0x558b320af130 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.273+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b3209e2e0 0x558b320a80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.273+0000 7f2712ff5640 0 client.26198 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.274+0000 7f273802b740 1 --2- 172.21.15.129:0/1336148832 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b320a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.274+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 >> 172.21.15.129:0/1336148832 conn(0x558b32099010 msgr2=0x558b3209a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.280+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 shutdown_connections 2024-09-05T01:27:40.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.282+0000 7f273802b740 1 -- 172.21.15.129:0/1336148832 wait complete. 2024-09-05T01:27:40.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.284+0000 7f273802b740 20 client.26198 trim_cache size 0 max 16384 2024-09-05T01:27:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.315+0000 7f273802b740 1 Processor -- start 2024-09-05T01:27:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.315+0000 7f273802b740 1 -- start start 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.315+0000 7f273802b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b3209dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.315+0000 7f273802b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 0x558b320af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.315+0000 7f273802b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f273802b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27301bc710 con 0x558b320a3370 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f273802b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b9fe0 con 0x558b3209e2e0 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f273802b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b7e60 con 0x558b3209d9a0 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b3209dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 0x558b320af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272effd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b3209dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:58734/0 (socket says 172.21.15.129:58734) 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272effd640 1 -- 172.21.15.129:0/2561232190 learned_addr learned my addr 172.21.15.129:0/2561232190 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272f7fe640 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 -- 172.21.15.129:0/2561232190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 msgr2=0x558b3209dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b3209dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 -- 172.21.15.129:0/2561232190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 msgr2=0x558b320993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 -- 172.21.15.129:0/2561232190 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b320a5810 con 0x558b3209e2e0 2024-09-05T01:27:40.320 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f272e7fc640 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 0x558b320af0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2724008000 tx=0x7f2724005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf2fc85b575a21f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.316+0000 7f2714ff9640 1 -- 172.21.15.129:0/2561232190 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724037040 con 0x558b3209e2e0 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.317+0000 7f2714ff9640 1 -- 172.21.15.129:0/2561232190 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2724004340 con 0x558b3209e2e0 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.317+0000 7f2714ff9640 1 -- 172.21.15.129:0/2561232190 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27240309b0 con 0x558b3209e2e0 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 -- 172.21.15.129:0/2561232190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 msgr2=0x558b320af0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 0x558b320af0f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2724008000 tx=0x7f2724005d70 comp rx=0 tx=0).stop 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 -- 172.21.15.129:0/2561232190 shutdown_connections 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209e2e0 0x558b320af0f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 --2- 172.21.15.129:0/2561232190 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209d9a0 0x558b3209dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.318+0000 7f273802b740 1 -- 172.21.15.129:0/2561232190 >> 172.21.15.129:0/2561232190 conn(0x558b32099010 msgr2=0x558b320a7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.321+0000 7f273802b740 1 -- 172.21.15.129:0/2561232190 shutdown_connections 2024-09-05T01:27:40.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.321+0000 7f273802b740 1 -- 172.21.15.129:0/2561232190 wait complete. 2024-09-05T01:27:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.321+0000 7f273802b740 1 -- start start 2024-09-05T01:27:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f273802b740 1 Processor -- start 2024-09-05T01:27:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f273802b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 0x558b320ac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f273802b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f272effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 0x558b320ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f272effd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 0x558b320ac100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:58714/0 (socket says 172.21.15.129:58714) 2024-09-05T01:27:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.322+0000 7f272effd640 1 -- 172.21.15.129:0/1352978954 learned_addr learned my addr 172.21.15.129:0/1352978954 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b84a0 con 0x558b320a3370 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b9910 con 0x558b3209d9a0 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f27301b8b70 con 0x558b3209e2e0 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f273802b740 10 client.0 _ll_register_callbacks cb 0x7f2728008810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 -- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 msgr2=0x558b320ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 0x558b320ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 -- 172.21.15.129:0/1352978954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 msgr2=0x558b320ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.323+0000 7f272e7fc640 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b320adc60 con 0x558b3209e2e0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.324+0000 7f272e7fc640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320ac640 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2724009780 tx=0x7f2724030d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b97449621d3976e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.324+0000 7f2712ff5640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.324+0000 7f2712ff5640 1 -- 172.21.15.129:0/1352978954 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724037040 con 0x558b3209e2e0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.324+0000 7f2712ff5640 1 -- 172.21.15.129:0/1352978954 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2724005240 con 0x558b3209e2e0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.324+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2724004340 con 0x558b3209e2e0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f2712ff5640 1 -- 172.21.15.129:0/1352978954 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2724056050 con 0x558b3209e2e0 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 10 client.26089 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({fsmap=34}) -- 0x558b320aee90 con 0x558b3209e2e0 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f2712ff5640 1 -- 172.21.15.129:0/1352978954 <== mon.1 v2:172.21.15.194:3300/0 4 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f27240046c0 con 0x558b3209e2e0 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 10 client.26089 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 10 client.26089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 20 client.26089 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b32120ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.325+0000 7f273802b740 4 client.26089 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.326+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2724004340 con 0x558b320b3cb0 2024-09-05T01:27:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.326+0000 7f272effd640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b32120ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.326+0000 7f2712ff5640 10 client.26089 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.326+0000 7f272effd640 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b32120ac0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2eb3b8577783ffd server_cookie=66d6e9b0b3b69040 in_seq=0 out_seq=0 2024-09-05T01:27:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.369+0000 7f2712ff5640 1 -- 172.21.15.129:0/1352978954 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2724004340 con 0x558b320b3cb0 2024-09-05T01:27:40.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.369+0000 7f2712ff5640 10 client.26089 handle_command_reply: tid=0 2024-09-05T01:27:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.370+0000 7f273802b740 1 client.26089 shutdown 2024-09-05T01:27:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.372+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 msgr2=0x558b320ac640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.372+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320ac640 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2724009780 tx=0x7f2724030d90 comp rx=0 tx=0).stop 2024-09-05T01:27:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 shutdown_connections 2024-09-05T01:27:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x558b320b3cb0 0x558b32120ac0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f2712ff5640 0 client.26089 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558b320a3370 0x558b320ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558b3209e2e0 0x558b320ac640 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 --2- 172.21.15.129:0/1352978954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558b3209d9a0 0x558b320ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.375+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 >> 172.21.15.129:0/1352978954 conn(0x558b32099010 msgr2=0x558b320979a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.376+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 shutdown_connections 2024-09-05T01:27:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.376+0000 7f273802b740 1 -- 172.21.15.129:0/1352978954 wait complete. 2024-09-05T01:27:40.380 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.378+0000 7f273802b740 20 client.26089 trim_cache size 0 max 16384 2024-09-05T01:27:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:27:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:27:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:27:40.381 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:27:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 msgr2=0x7f27180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 0x7f27180a5260 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2728012710 tx=0x7f2728004ab0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.387 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 msgr2=0x7f273019e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9a0 tx=0x7f272000ee60 comp rx=0 tx=0).stop 2024-09-05T01:27:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 shutdown_connections 2024-09-05T01:27:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f27180a2da0 0x7f27180a5260 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f27301a3ad0 0x7f27301a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f27301025f0 0x7f273019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 --2- 172.21.15.129:0/1598554249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2730100a40 0x7f273019e530 secure :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f272000e9a0 tx=0x7f272000ee60 comp rx=0 tx=0).stop 2024-09-05T01:27:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.384+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 >> 172.21.15.129:0/1598554249 conn(0x7f27300fc1e0 msgr2=0x7f2730103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.385+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 shutdown_connections 2024-09-05T01:27:40.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:40.385+0000 7f273528d640 1 -- 172.21.15.129:0/1598554249 wait complete. 2024-09-05T01:27:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:40 smithi183 ceph-mon[34589]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 894 op/s 2024-09-05T01:27:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:40 smithi129 ceph-mon[30979]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 894 op/s 2024-09-05T01:27:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:40 smithi194 ceph-mon[34941]: pgmap v724: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 894 op/s 2024-09-05T01:27:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:42 smithi183 ceph-mon[34589]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-09-05T01:27:42.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:42 smithi129 ceph-mon[30979]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-09-05T01:27:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:42 smithi194 ceph-mon[34941]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-09-05T01:27:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:44 smithi183 ceph-mon[34589]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-09-05T01:27:44.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:44 smithi129 ceph-mon[30979]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-09-05T01:27:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:44 smithi194 ceph-mon[34941]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2024-09-05T01:27:45.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 640.0 s, 605.5 tps, lat 155.925 ms stddev 143.662 2024-09-05T01:27:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:46 smithi183 ceph-mon[34589]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-05T01:27:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:46 smithi129 ceph-mon[30979]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-05T01:27:46.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:46 smithi194 ceph-mon[34941]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 11 MiB/s wr, 938 op/s 2024-09-05T01:27:46.510 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-09-05T01:27:46.510 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:27:46.854 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 -- 172.21.15.129:0/1962050873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 msgr2=0x7f1ff80a4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 --2- 172.21.15.129:0/1962050873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff80a4e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2000067960 tx=0x7f2000092750 comp rx=0 tx=0).stop 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 -- 172.21.15.129:0/1962050873 shutdown_connections 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 --2- 172.21.15.129:0/1962050873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80acb90 0x7f1ff80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 --2- 172.21.15.129:0/1962050873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80a5430 0x7f1ff80ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.538 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 --2- 172.21.15.129:0/1962050873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff80a4e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 -- 172.21.15.129:0/1962050873 >> 172.21.15.129:0/1962050873 conn(0x7f1ff801a0e0 msgr2=0x7f1ff801a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 -- 172.21.15.129:0/1962050873 shutdown_connections 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.535+0000 7f2005cd1640 1 -- 172.21.15.129:0/1962050873 wait complete. 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 Processor -- start 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 -- start start 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff81433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 0x7f1ff8147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4610 con 0x7f1ff80a5430 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4490 con 0x7f1ff80a4a80 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.536+0000 7f2005cd1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff80b4790 con 0x7f1ff80acb90 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.537+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:47.539 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.537+0000 7f2004ccf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff81433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:47.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.537+0000 7f1ffffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55340/0 (socket says 172.21.15.129:55340) 2024-09-05T01:27:47.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.537+0000 7f1ffffff640 1 -- 172.21.15.129:0/3317863481 learned_addr learned my addr 172.21.15.129:0/3317863481 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:47.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.537+0000 7f20054d0640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 0x7f1ff8147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 -- 172.21.15.129:0/3317863481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 msgr2=0x7f1ff8147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 0x7f1ff8147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 -- 172.21.15.129:0/3317863481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 msgr2=0x7f1ff81433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff81433a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 -- 172.21.15.129:0/3317863481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8148370 con 0x7f1ff80a5430 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f1ffffff640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f2004ccf640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff81433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.538+0000 7f20054d0640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 0x7f1ff8147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.539+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00cd80 con 0x7f1ff80a5430 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.539+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fec0108e0 con 0x7f1ff80a5430 2024-09-05T01:27:47.541 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.539+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec018880 con 0x7f1ff80a5430 2024-09-05T01:27:47.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.541+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff8148660 con 0x7f1ff80a5430 2024-09-05T01:27:47.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.541+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff813a8f0 con 0x7f1ff80a5430 2024-09-05T01:27:47.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.542+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff80a5ac0 con 0x7f1ff80a5430 2024-09-05T01:27:47.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.545+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1fec010a90 con 0x7f1ff80a5430 2024-09-05T01:27:47.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.546+0000 7f1ffdffb640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:47.547 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.546+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1fec0cf9a0 con 0x7f1ff80a5430 2024-09-05T01:27:47.548 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.547+0000 7f2004ccf640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:47.549 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.549+0000 7f2004ccf640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 0x7f1ff00a5050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20000680a0 tx=0x7f200009c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:47.557 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.556+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fec09c4f0 con 0x7f1ff80a5430 2024-09-05T01:27:47.952 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.951+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ff8144730 con 0x7f1ff80a5430 2024-09-05T01:27:47.953 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:27:47.954 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.953+0000 7f1ffdffb640 1 -- 172.21.15.129:0/3317863481 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f1ff8144730 con 0x7f1ff80a5430 2024-09-05T01:27:47.954 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:27:47.954 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:27:47.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.956+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 msgr2=0x7f1ff00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:47.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.956+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 0x7f1ff00a5050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f20000680a0 tx=0x7f200009c040 comp rx=0 tx=0).stop 2024-09-05T01:27:47.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 msgr2=0x7f1ff81438e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:47.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00e9f0 tx=0x7f1fec00eeb0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 shutdown_connections 2024-09-05T01:27:47.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1ff00a2b90 0x7f1ff00a5050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1ff80acb90 0x7f1ff8147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1ff80a5430 0x7f1ff81438e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 --2- 172.21.15.129:0/3317863481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ff80a4a80 0x7f1ff81433a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:47.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.957+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 >> 172.21.15.129:0/3317863481 conn(0x7f1ff801a0e0 msgr2=0x7f1ff80aa720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:47.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.960+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 shutdown_connections 2024-09-05T01:27:47.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:47.960+0000 7f2005cd1640 1 -- 172.21.15.129:0/3317863481 wait complete. 2024-09-05T01:27:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:48 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3317863481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:48 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3317863481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:48 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3317863481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:27:49.278 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:49 smithi129 ceph-mon[30979]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-09-05T01:27:49.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:49 smithi183 ceph-mon[34589]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-09-05T01:27:49.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:49 smithi194 ceph-mon[34941]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-09-05T01:27:49.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0' 2024-09-05T01:27:49.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 146.4 --await-for 45.0 2024-09-05T01:27:50.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 645.0 s, 688.6 tps, lat 156.433 ms stddev 210.916 2024-09-05T01:27:50.220 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:27:50.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:50 smithi183 ceph-mon[34589]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 910 op/s 2024-09-05T01:27:50.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:50 smithi129 ceph-mon[30979]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 910 op/s 2024-09-05T01:27:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:50 smithi194 ceph-mon[34941]: pgmap v729: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 10 MiB/s wr, 910 op/s 2024-09-05T01:27:50.846 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.844+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/2946913452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 msgr2=0x7f9c741095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.844+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/2946913452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c741095d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009860 tx=0x7f9c7002a340 comp rx=0 tx=0).stop 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/2946913452 shutdown_connections 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/2946913452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c7410e7f0 0x7f9c7410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/2946913452 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c74109b10 0x7f9c7410e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/2946913452 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c741095d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/2946913452 >> 172.21.15.129:0/2946913452 conn(0x7f9c740735f0 msgr2=0x7f9c74073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/2946913452 shutdown_connections 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.845+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/2946913452 wait complete. 2024-09-05T01:27:50.847 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.846+0000 7f9c7ba7b640 1 Processor -- start 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.846+0000 7f9c7ba7b640 1 -- start start 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.846+0000 7f9c7ba7b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.846+0000 7f9c7ba7b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c74109b10 0x7f9c741178c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.846+0000 7f9c7ba7b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c7410e7f0 0x7f9c741a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c7ba7b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7411a490 con 0x7f9c74109b10 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c7ba7b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7411a310 con 0x7f9c7410e7f0 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c7ba7b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c7411a610 con 0x7f9c741091f0 2024-09-05T01:27:50.848 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:54222/0 (socket says 172.21.15.129:54222) 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 -- 172.21.15.129:0/97190291 learned_addr learned my addr 172.21.15.129:0/97190291 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 -- 172.21.15.129:0/97190291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c7410e7f0 msgr2=0x7f9c741a2f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c7410e7f0 0x7f9c741a2f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 -- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c74109b10 msgr2=0x7f9c741178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c74109b10 0x7f9c741178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.849 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 -- 172.21.15.129:0/97190291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c741a3630 con 0x7f9c741091f0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.847+0000 7f9c79817640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c700057c0 tx=0x7f9c70030d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.848+0000 7f9c6affd640 1 -- 172.21.15.129:0/97190291 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70037050 con 0x7f9c741091f0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.848+0000 7f9c6affd640 1 -- 172.21.15.129:0/97190291 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70043070 con 0x7f9c741091f0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.849+0000 7f9c6affd640 1 -- 172.21.15.129:0/97190291 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c70003cd0 con 0x7f9c741091f0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.849+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c741a38c0 con 0x7f9c741091f0 2024-09-05T01:27:50.850 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.849+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7419fe80 con 0x7f9c741091f0 2024-09-05T01:27:50.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.852+0000 7f9c6affd640 1 -- 172.21.15.129:0/97190291 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9c70030450 con 0x7f9c741091f0 2024-09-05T01:27:50.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.853+0000 7f9c6affd640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 0x7f9c600a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.853+0000 7f9c6affd640 1 -- 172.21.15.129:0/97190291 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9c700ef340 con 0x7f9c741091f0 2024-09-05T01:27:50.857 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.856+0000 7f9c79016640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 0x7f9c600a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.858 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.857+0000 7f9c79016640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 0x7f9c600a5210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c640058c0 tx=0x7f9c64005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 Processor -- start 2024-09-05T01:27:50.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 -- start start 2024-09-05T01:27:50.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c2c68fc2a0 con 0x55c2c6996900 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c68a3b10 con 0x55c2c69a1ae0 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.863+0000 7f9c7de54740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c2c68f8260 con 0x55c2c6997220 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55358/0 (socket says 172.21.15.129:55358) 2024-09-05T01:27:50.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 -- 172.21.15.129:0/3882834778 learned_addr learned my addr 172.21.15.129:0/3882834778 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c7a018640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79016640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 -- 172.21.15.129:0/3882834778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 msgr2=0x55c2c69a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 -- 172.21.15.129:0/3882834778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 msgr2=0x55c2c69a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c79817640 1 -- 172.21.15.129:0/3882834778 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c666db90 con 0x55c2c6996900 2024-09-05T01:27:50.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.865+0000 7f9c7a018640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:27:50.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.866+0000 7f9c79817640 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9c700c3ef0 tx=0x7f9c7002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2f2bb607021c6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.867+0000 7f9c5ffff640 1 -- 172.21.15.129:0/3882834778 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700d4070 con 0x55c2c6996900 2024-09-05T01:27:50.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.867+0000 7f9c5ffff640 1 -- 172.21.15.129:0/3882834778 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70038040 con 0x55c2c6996900 2024-09-05T01:27:50.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.868+0000 7f9c5ffff640 1 -- 172.21.15.129:0/3882834778 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700bb7b0 con 0x55c2c6996900 2024-09-05T01:27:50.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.868+0000 7f9c7de54740 1 -- 172.21.15.129:0/3882834778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 msgr2=0x55c2c6996ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:50.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.868+0000 7f9c7de54740 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9c700c3ef0 tx=0x7f9c7002a340 comp rx=0 tx=0).stop 2024-09-05T01:27:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 -- 172.21.15.129:0/3882834778 shutdown_connections 2024-09-05T01:27:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 --2- 172.21.15.129:0/3882834778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c6996ce0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 -- 172.21.15.129:0/3882834778 >> 172.21.15.129:0/3882834778 conn(0x55c2c6992e90 msgr2=0x55c2c6993270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:50.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 -- 172.21.15.129:0/3882834778 shutdown_connections 2024-09-05T01:27:50.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.869+0000 7f9c7de54740 1 -- 172.21.15.129:0/3882834778 wait complete. 2024-09-05T01:27:50.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 -- start start 2024-09-05T01:27:50.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 Processor -- start 2024-09-05T01:27:50.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c699e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6997220 0x55c2c699eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c2c69a4470 con 0x55c2c6997220 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c2c68846e0 con 0x55c2c6996900 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55c2c69a45f0 con 0x55c2c69a1ae0 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.870+0000 7f9c7de54740 10 client.0 _ll_register_callbacks cb 0x55c2c68a8a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:50.872 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c79817640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c699e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:54254/0 (socket says 172.21.15.129:54254) 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 -- 172.21.15.129:0/272065745 learned_addr learned my addr 172.21.15.129:0/272065745 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c79016640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6997220 0x55c2c699eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 -- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 msgr2=0x55c2c699e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c699e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 -- 172.21.15.129:0/272065745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6997220 msgr2=0x55c2c699eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.873 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6997220 0x55c2c699eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.871+0000 7f9c7a018640 1 -- 172.21.15.129:0/272065745 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c69ad9b0 con 0x55c2c69a1ae0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c7a018640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010ef0 tx=0x7f9c6c00e760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aad90a887336dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c009e10 con 0x55c2c69a1ae0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9c6c004b20 con 0x55c2c69a1ae0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c70038040 con 0x55c2c69a1ae0 2024-09-05T01:27:50.874 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.872+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c6c022050 con 0x55c2c69a1ae0 2024-09-05T01:27:50.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.874+0000 7f9c7de54740 10 client.26104 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:50.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.874+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2c69b0340 con 0x55c2c69a1ae0 2024-09-05T01:27:50.875 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.874+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c70038040 con 0x55c2c69a1ae0 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9c6c00b040 con 0x55c2c69a1ae0 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 10 client.26104 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 10 client.26104 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 20 client.26104 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c69b1850 0x55c2c69b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 4 client.26104 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:50.876 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.875+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c69099d0 con 0x55c2c69b1850 2024-09-05T01:27:50.877 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.876+0000 7f9c79817640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c69b1850 0x55c2c69b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.878+0000 7f9c79817640 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c69b1850 0x55c2c69b3c70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb8b01e2f8397f7 server_cookie=8dda1e1e9e78ad5d in_seq=0 out_seq=0 2024-09-05T01:27:50.879 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.879+0000 7f9c5d7fa640 10 client.26104 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:50.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/272065745 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2c69099d0 con 0x55c2c69b1850 2024-09-05T01:27:50.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c5d7fa640 10 client.26104 handle_command_reply: tid=0 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 client.26104 shutdown 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 msgr2=0x55c2c699f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010ef0 tx=0x7f9c6c00e760 comp rx=0 tx=0).stop 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 shutdown_connections 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c69b1850 0x55c2c69b3c70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c699f430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6997220 0x55c2c699eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 --2- 172.21.15.129:0/272065745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c699e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 >> 172.21.15.129:0/272065745 conn(0x55c2c6992e90 msgr2=0x55c2c6992000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.880+0000 7f9c5d7fa640 0 client.26104 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.881+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 shutdown_connections 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.881+0000 7f9c7de54740 1 -- 172.21.15.129:0/272065745 wait complete. 2024-09-05T01:27:50.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.881+0000 7f9c7de54740 20 client.26104 trim_cache size 0 max 16384 2024-09-05T01:27:50.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 Processor -- start 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 -- start start 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c699f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6997220 0x55c2c6998a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c69b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c8bb0 con 0x55c2c6996900 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c6480 con 0x55c2c6997220 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c7de54740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c42d0 con 0x55c2c69a1ae0 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.915+0000 7f9c79817640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c699f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c79016640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6997220 0x55c2c6998a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c79817640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c699f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55382/0 (socket says 172.21.15.129:55382) 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c79817640 1 -- 172.21.15.129:0/2905338205 learned_addr learned my addr 172.21.15.129:0/2905338205 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:50.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c69b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 -- 172.21.15.129:0/2905338205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6997220 msgr2=0x55c2c6998a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6997220 0x55c2c6998a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 -- 172.21.15.129:0/2905338205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 msgr2=0x55c2c699f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c699f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.916+0000 7f9c7a018640 1 -- 172.21.15.129:0/2905338205 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c6996730 con 0x55c2c69a1ae0 2024-09-05T01:27:50.918 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.917+0000 7f9c7a018640 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c69b9540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010ec0 tx=0x7f9c6c0107f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a408f934024756 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.918+0000 7f9c5f7fe640 1 -- 172.21.15.129:0/2905338205 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c02b070 con 0x55c2c69a1ae0 2024-09-05T01:27:50.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.918+0000 7f9c5f7fe640 1 -- 172.21.15.129:0/2905338205 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c6c004e10 con 0x55c2c69a1ae0 2024-09-05T01:27:50.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.918+0000 7f9c5f7fe640 1 -- 172.21.15.129:0/2905338205 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6c026430 con 0x55c2c69a1ae0 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 -- 172.21.15.129:0/2905338205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 msgr2=0x55c2c69b9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c69b9540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c010ec0 tx=0x7f9c6c0107f0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 -- 172.21.15.129:0/2905338205 shutdown_connections 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c69a1ae0 0x55c2c69b9540 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6997220 0x55c2c6998a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 --2- 172.21.15.129:0/2905338205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c6996900 0x55c2c699f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.920+0000 7f9c7de54740 1 -- 172.21.15.129:0/2905338205 >> 172.21.15.129:0/2905338205 conn(0x55c2c6992e90 msgr2=0x55c2c6998350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.921+0000 7f9c7de54740 1 -- 172.21.15.129:0/2905338205 shutdown_connections 2024-09-05T01:27:50.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.921+0000 7f9c7de54740 1 -- 172.21.15.129:0/2905338205 wait complete. 2024-09-05T01:27:50.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.922+0000 7f9c7de54740 1 -- start start 2024-09-05T01:27:50.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.922+0000 7f9c7de54740 1 Processor -- start 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.922+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c69aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.922+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7de54740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7de54740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c4940 con 0x55c2c69a1ae0 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7de54740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c5db0 con 0x55c2c6996900 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7de54740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9c741c5010 con 0x55c2c6997220 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7de54740 10 client.0 _ll_register_callbacks cb 0x7f9c6c010b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7a018640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7a018640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55394/0 (socket says 172.21.15.129:55394) 2024-09-05T01:27:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.923+0000 7f9c7a018640 1 -- 172.21.15.129:0/1575007006 learned_addr learned my addr 172.21.15.129:0/1575007006 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c69aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 -- 172.21.15.129:0/1575007006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 msgr2=0x55c2c69a5470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a5470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 -- 172.21.15.129:0/1575007006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 msgr2=0x55c2c69a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 -- 172.21.15.129:0/1575007006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c6999770 con 0x55c2c6996900 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c79817640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c69aa140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c700bb460 tx=0x7f9c700bb3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f29d1a3685fb291 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.924+0000 7f9c5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:27:50.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.925+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700d4040 con 0x55c2c6996900 2024-09-05T01:27:50.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.925+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9c700ecae0 con 0x55c2c6996900 2024-09-05T01:27:50.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.925+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c6c004e10 con 0x55c2c6996900 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.925+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c700328c0 con 0x55c2c6996900 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.926+0000 7f9c7de54740 10 client.26219 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.926+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2c6a1c480 con 0x55c2c6996900 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.926+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9c700bbeb0 con 0x55c2c6996900 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9c70030740 con 0x55c2c6996900 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 10 client.26219 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 10 client.26219 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 20 client.26219 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c6a1ca10 0x55c2c6a1ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 4 client.26219 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}] 2024-09-05T01:27:50.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.4, "await_for": 45.0}) -- 0x7f9c6c004e10 con 0x55c2c6a1ca10 2024-09-05T01:27:50.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.927+0000 7f9c79016640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c6a1ca10 0x55c2c6a1ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:50.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.928+0000 7f9c5d7fa640 10 client.26219 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:50.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:50.928+0000 7f9c79016640 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c6a1ca10 0x55c2c6a1ee00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f817397599358db server_cookie=7f9f86cd0619eb0f in_seq=0 out_seq=0 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.240+0000 7f9c5d7fa640 1 -- 172.21.15.129:0/1575007006 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f9c6c004e10 con 0x55c2c6a1ca10 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.240+0000 7f9c5d7fa640 10 client.26219 handle_command_reply: tid=0 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.240+0000 7f9c7de54740 1 client.26219 shutdown 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.240+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 msgr2=0x55c2c69aa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.240+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c69aa140 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9c700bb460 tx=0x7f9c700bb3f0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 shutdown_connections 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55c2c6a1ca10 0x55c2c6a1ee00 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c2c69a1ae0 0x55c2c69a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55c2c6997220 0x55c2c69a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 --2- 172.21.15.129:0/1575007006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c2c6996900 0x55c2c69aa140 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c5d7fa640 0 client.26219 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:51.242 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 >> 172.21.15.129:0/1575007006 conn(0x55c2c6992e90 msgr2=0x55c2c6998b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 shutdown_connections 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 1 -- 172.21.15.129:0/1575007006 wait complete. 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7de54740 20 client.26219 trim_cache size 0 max 16384 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 46, 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "eeee04fa": { 2024-09-05T01:27:51.243 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 46, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 146.4, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:27:51.244 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:27:51.245 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 msgr2=0x7f9c600a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 0x7f9c600a5210 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9c640058c0 tx=0x7f9c64005850 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 msgr2=0x7f9c74117380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9c700057c0 tx=0x7f9c70030d80 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 shutdown_connections 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.241+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9c600a2d50 0x7f9c600a5210 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9c7410e7f0 0x7f9c741a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9c74109b10 0x7f9c741178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 --2- 172.21.15.129:0/97190291 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9c741091f0 0x7f9c74117380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 >> 172.21.15.129:0/97190291 conn(0x7f9c740735f0 msgr2=0x7f9c7410c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 shutdown_connections 2024-09-05T01:27:51.246 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:27:51.242+0000 7f9c7ba7b640 1 -- 172.21.15.129:0/97190291 wait complete. 2024-09-05T01:27:51.928 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eeee04fa', quiesce took 5.4 seconds. Will release after: 21.0 2024-09-05T01:27:52.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:52 smithi129 ceph-mon[30979]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-09-05T01:27:52.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:52 smithi183 ceph-mon[34589]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-09-05T01:27:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:52 smithi194 ceph-mon[34941]: pgmap v730: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 10 MiB/s wr, 868 op/s 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 -- 172.21.15.129:0/3207890826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 msgr2=0x7f1df81020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/3207890826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df81020d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1dec009900 tx=0x7f1dec02a410 comp rx=0 tx=0).stop 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 -- 172.21.15.129:0/3207890826 shutdown_connections 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/3207890826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8102610 0x7f1df8108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/3207890826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df81020d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/3207890826 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8100a60 0x7f1df8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 -- 172.21.15.129:0/3207890826 >> 172.21.15.129:0/3207890826 conn(0x7f1df80fc1e0 msgr2=0x7f1df80fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 -- 172.21.15.129:0/3207890826 shutdown_connections 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.491+0000 7f1dfe89b640 1 -- 172.21.15.129:0/3207890826 wait complete. 2024-09-05T01:27:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.492+0000 7f1dfe89b640 1 Processor -- start 2024-09-05T01:27:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.492+0000 7f1dfe89b640 1 -- start start 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.492+0000 7f1dfe89b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8102610 0x7f1df819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810c5c0 con 0x7f1df8101c50 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810c440 con 0x7f1df8100a60 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df810c740 con 0x7f1df8102610 2024-09-05T01:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40760/0 (socket says 172.21.15.129:40760) 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41576/0 (socket says 172.21.15.129:41576) 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 -- 172.21.15.129:0/1315525602 learned_addr learned my addr 172.21.15.129:0/1315525602 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfce38640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8102610 0x7f1df819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 -- 172.21.15.129:0/1315525602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8102610 msgr2=0x7f1df819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8102610 0x7f1df819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 -- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 msgr2=0x7f1df819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 -- 172.21.15.129:0/1315525602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df819f910 con 0x7f1df8100a60 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df77fe640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df7fff640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d950 tx=0x7f1de800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df57fa640 1 -- 172.21.15.129:0/1315525602 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8017020 con 0x7f1df8100a60 2024-09-05T01:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df819fc00 con 0x7f1df8100a60 2024-09-05T01:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df57fa640 1 -- 172.21.15.129:0/1315525602 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1de8009c30 con 0x7f1df8100a60 2024-09-05T01:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.493+0000 7f1df57fa640 1 -- 172.21.15.129:0/1315525602 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de800f620 con 0x7f1df8100a60 2024-09-05T01:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.494+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81a0120 con 0x7f1df8100a60 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 Processor -- start 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1df57fa640 1 -- 172.21.15.129:0/1315525602 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1de801a020 con 0x7f1df8100a60 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 -- start start 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f3f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f3feda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55841f35d2f0 con 0x55841f3ff370 2024-09-05T01:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55841f305270 con 0x55841f3fa2e0 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1e01633740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55841f3592b0 con 0x55841f3f99a0 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41590/0 (socket says 172.21.15.129:41590) 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.496+0000 7f1df77fe640 1 -- 172.21.15.129:0/2615655277 learned_addr learned my addr 172.21.15.129:0/2615655277 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dfce38640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f3f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df7fff640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f3feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 -- 172.21.15.129:0/2615655277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 msgr2=0x55841f3f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f3f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 -- 172.21.15.129:0/2615655277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 msgr2=0x55841f3feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f3feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 -- 172.21.15.129:0/2615655277 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841f304340 con 0x55841f3ff370 2024-09-05T01:27:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dfce38640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f3f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df57fa640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 0x7f1de00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dfce38640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 0x7f1de00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df57fa640 1 -- 172.21.15.129:0/1315525602 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1de80cf360 con 0x7f1df8100a60 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1dec02aa00 tx=0x7f1dec004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9326671aafdebfa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dde7fc640 1 -- 172.21.15.129:0/2615655277 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec040070 con 0x55841f3ff370 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dde7fc640 1 -- 172.21.15.129:0/2615655277 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec003d20 con 0x55841f3ff370 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.497+0000 7f1dde7fc640 1 -- 172.21.15.129:0/2615655277 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec035040 con 0x55841f3ff370 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- 172.21.15.129:0/2615655277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 msgr2=0x55841f401830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1dec02aa00 tx=0x7f1dec004520 comp rx=0 tx=0).stop 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1dfce38640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 0x7f1de00a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005fd0 tx=0x7f1df0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- 172.21.15.129:0/2615655277 shutdown_connections 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f401830 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f3feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 --2- 172.21.15.129:0/2615655277 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f3f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- 172.21.15.129:0/2615655277 >> 172.21.15.129:0/2615655277 conn(0x55841f3f5010 msgr2=0x55841f3f5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- 172.21.15.129:0/2615655277 shutdown_connections 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- 172.21.15.129:0/2615655277 wait complete. 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.498+0000 7f1e01633740 1 -- start start 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 Processor -- start 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f403b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3ff370 0x55841f40b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55841f4022e0 con 0x55841f3fa2e0 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55841f2e6290 con 0x55841f3f99a0 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55841f402460 con 0x55841f3ff370 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1e01633740 10 client.0 _ll_register_callbacks cb 0x55841f317140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41592/0 (socket says 172.21.15.129:41592) 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 -- 172.21.15.129:0/2708964766 learned_addr learned my addr 172.21.15.129:0/2708964766 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df7fff640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f403b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1dfce38640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3ff370 0x55841f40b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 -- 172.21.15.129:0/2708964766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3ff370 msgr2=0x55841f40b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3ff370 0x55841f40b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 -- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 msgr2=0x55841f403b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f403b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 -- 172.21.15.129:0/2708964766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f070210 con 0x55841f3fa2e0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1df77fe640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005ab0 tx=0x7f1dec032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc634775461e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.499+0000 7f1dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec035070 con 0x55841f3fa2e0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1dec003670 con 0x55841f3fa2e0 2024-09-05T01:27:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec003d20 con 0x55841f3fa2e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1dec056050 con 0x55841f3fa2e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 10 client.16662 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55841f13c620 con 0x55841f3fa2e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1dec003d20 con 0x55841f3fa2e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1dec004750 con 0x55841f3fa2e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 10 client.16662 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 20 client.16662 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f40f3e0 0x55841f4117d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 4 client.16662 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.500+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55841f36aa20 con 0x55841f40f3e0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.501+0000 7f1df7fff640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f40f3e0 0x55841f4117d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.501+0000 7f1dd77fe640 10 client.16662 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.501+0000 7f1df7fff640 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f40f3e0 0x55841f4117d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70c3b7f0bfeeed8 server_cookie=1bebcdd53c7e1e82 in_seq=0 out_seq=0 2024-09-05T01:27:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1dd77fe640 1 -- 172.21.15.129:0/2708964766 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55841f36aa20 con 0x55841f40f3e0 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1dd77fe640 10 client.16662 handle_command_reply: tid=0 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 client.16662 shutdown 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 msgr2=0x55841f4040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1dec005ab0 tx=0x7f1dec032ef0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 shutdown_connections 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f40f3e0 0x55841f4117d0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3ff370 0x55841f40b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3fa2e0 0x55841f4040a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 --2- 172.21.15.129:0/2708964766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f403b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 >> 172.21.15.129:0/2708964766 conn(0x55841f3f5010 msgr2=0x55841f3f6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1dd77fe640 0 client.16662 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 shutdown_connections 2024-09-05T01:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 1 -- 172.21.15.129:0/2708964766 wait complete. 2024-09-05T01:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.502+0000 7f1e01633740 20 client.16662 trim_cache size 0 max 16384 2024-09-05T01:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.516+0000 7f1e01633740 1 Processor -- start 2024-09-05T01:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 -- start start 2024-09-05T01:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f3f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3fa2e0 0x55841f3f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81ad060 con 0x55841f3ff370 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81aa930 con 0x55841f3f99a0 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1e01633740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a8780 con 0x55841f3fa2e0 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41594/0 (socket says 172.21.15.129:41594) 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 -- 172.21.15.129:0/3305375777 learned_addr learned my addr 172.21.15.129:0/3305375777 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1df7fff640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f3f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1df77fe640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3fa2e0 0x55841f3f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 -- 172.21.15.129:0/3305375777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3fa2e0 msgr2=0x55841f3f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3fa2e0 0x55841f3f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 -- 172.21.15.129:0/3305375777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 msgr2=0x55841f3f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f3f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 -- 172.21.15.129:0/3305375777 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841f3f4e30 con 0x55841f3ff370 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1df7fff640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f3f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dfce38640 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1df0003d10 tx=0x7f1df0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0731213a3869a71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dddffb640 1 -- 172.21.15.129:0/3305375777 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0004540 con 0x55841f3ff370 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dddffb640 1 -- 172.21.15.129:0/3305375777 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df00046e0 con 0x55841f3ff370 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.517+0000 7f1dddffb640 1 -- 172.21.15.129:0/3305375777 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df001f450 con 0x55841f3ff370 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- 172.21.15.129:0/3305375777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 msgr2=0x55841f3f5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1df0003d10 tx=0x7f1df0017000 comp rx=0 tx=0).stop 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- 172.21.15.129:0/3305375777 shutdown_connections 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f3f5e70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3fa2e0 0x55841f3f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 --2- 172.21.15.129:0/3305375777 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3f99a0 0x55841f3f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- 172.21.15.129:0/3305375777 >> 172.21.15.129:0/3305375777 conn(0x55841f3f5010 msgr2=0x55841f403b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- 172.21.15.129:0/3305375777 shutdown_connections 2024-09-05T01:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- 172.21.15.129:0/3305375777 wait complete. 2024-09-05T01:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.518+0000 7f1e01633740 1 -- start start 2024-09-05T01:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 Processor -- start 2024-09-05T01:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f474e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f4753b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a8df0 con 0x55841f3ff370 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81aa260 con 0x55841f3fa2e0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a94c0 con 0x55841f3f99a0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1e01633740 10 client.0 _ll_register_callbacks cb 0x7f1dec0043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41596/0 (socket says 172.21.15.129:41596) 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 -- 172.21.15.129:0/906477266 learned_addr learned my addr 172.21.15.129:0/906477266 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1df77fe640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f4753b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1df7fff640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f474e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 -- 172.21.15.129:0/906477266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 msgr2=0x55841f474e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f474e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 -- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 msgr2=0x55841f4753b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f4753b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.519+0000 7f1dfce38640 1 -- 172.21.15.129:0/906477266 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f47ca00 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dfce38640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1df0012dc0 tx=0x7f1df000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8655c75aba4428 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0037060 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1df000f480 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df00046e0 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1df000f680 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 10 client.16668 fetch_fsmap learned FSMap version 34 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55841f47dcf0 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1df00046e0 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1df0005890 con 0x55841f3ff370 2024-09-05T01:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 10 client.16668 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 20 client.16668 populate_metadata read hostname 'smithi129' 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f47e6a0 0x55841f480ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 4 client.16668 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.520+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55841f364d70 con 0x55841f47e6a0 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.521+0000 7f1df7fff640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f47e6a0 0x55841f480ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.521+0000 7f1dd77fe640 10 client.16668 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.521+0000 7f1df7fff640 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f47e6a0 0x55841f480ac0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ef61018dd81fc2 server_cookie=d2402d949de39294 in_seq=0 out_seq=0 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.544+0000 7f1dd77fe640 1 -- 172.21.15.129:0/906477266 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55841f364d70 con 0x55841f47e6a0 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.544+0000 7f1dd77fe640 10 client.16668 handle_command_reply: tid=0 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.544+0000 7f1e01633740 1 client.16668 shutdown 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.544+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 msgr2=0x55841f47c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.544+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1df0012dc0 tx=0x7f1df000fc30 comp rx=0 tx=0).stop 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 shutdown_connections 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55841f47e6a0 0x55841f480ac0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55841f3ff370 0x55841f47c3f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55841f3fa2e0 0x55841f4753b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dd77fe640 0 client.16668 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 --2- 172.21.15.129:0/906477266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55841f3f99a0 0x55841f474e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 >> 172.21.15.129:0/906477266 conn(0x55841f3f5010 msgr2=0x55841f3f39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 shutdown_connections 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 1 -- 172.21.15.129:0/906477266 wait complete. 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1e01633740 20 client.16668 trim_cache size 0 max 16384 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:27:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 msgr2=0x7f1de00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 0x7f1de00a50f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1df0005fd0 tx=0x7f1df0009500 comp rx=0 tx=0).stop 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 msgr2=0x7f1df806aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1de800d950 tx=0x7f1de800de10 comp rx=0 tx=0).stop 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 shutdown_connections 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1de00a2c30 0x7f1de00a50f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1df8102610 0x7f1df819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1df8101c50 0x7f1df819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 --2- 172.21.15.129:0/1315525602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1df8100a60 0x7f1df806aeb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 >> 172.21.15.129:0/1315525602 conn(0x7f1df80fc1e0 msgr2=0x7f1df8104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 shutdown_connections 2024-09-05T01:27:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:27:53.545+0000 7f1dfe89b640 1 -- 172.21.15.129:0/1315525602 wait complete. 2024-09-05T01:27:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:54 smithi183 ceph-mon[34589]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 813 op/s 2024-09-05T01:27:54.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:54 smithi129 ceph-mon[30979]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 813 op/s 2024-09-05T01:27:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:54 smithi194 ceph-mon[34941]: pgmap v731: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 9.2 MiB/s wr, 813 op/s 2024-09-05T01:27:55.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 650.0 s, 99.1 tps, lat 152.781 ms stddev 46.530 2024-09-05T01:27:56.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:56 smithi183 ceph-mon[34589]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 651 op/s 2024-09-05T01:27:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:56 smithi129 ceph-mon[30979]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 651 op/s 2024-09-05T01:27:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:56 smithi194 ceph-mon[34941]: pgmap v732: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.2 MiB/s wr, 651 op/s 2024-09-05T01:27:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:27:58 smithi183 ceph-mon[34589]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-05T01:27:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:27:58 smithi129 ceph-mon[30979]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-05T01:27:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:27:58 smithi194 ceph-mon[34941]: pgmap v733: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 6.3 MiB/s wr, 574 op/s 2024-09-05T01:28:00.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 655.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:28:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:00 smithi183 ceph-mon[34589]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-09-05T01:28:00.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:00 smithi129 ceph-mon[30979]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-09-05T01:28:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:00 smithi194 ceph-mon[34941]: pgmap v734: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-09-05T01:28:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:02 smithi183 ceph-mon[34589]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-05T01:28:02.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:02 smithi129 ceph-mon[30979]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-05T01:28:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:02 smithi194 ceph-mon[34941]: pgmap v735: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.7 MiB/s wr, 267 op/s 2024-09-05T01:28:04.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:04 smithi183 ceph-mon[34589]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-05T01:28:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:04 smithi129 ceph-mon[30979]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-05T01:28:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:04 smithi194 ceph-mon[34941]: pgmap v736: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-09-05T01:28:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 660.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:28:06.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:06 smithi183 ceph-mon[34589]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 298 KiB/s wr, 47 op/s 2024-09-05T01:28:06.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:06 smithi129 ceph-mon[30979]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 298 KiB/s wr, 47 op/s 2024-09-05T01:28:06.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:06 smithi194 ceph-mon[34941]: pgmap v737: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 298 KiB/s wr, 47 op/s 2024-09-05T01:28:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 -- 172.21.15.129:0/1153877243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 msgr2=0x7f026c1073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 --2- 172.21.15.129:0/1153877243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1073a0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0254009900 tx=0x7f025402a410 comp rx=0 tx=0).stop 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 -- 172.21.15.129:0/1153877243 shutdown_connections 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 --2- 172.21.15.129:0/1153877243 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 --2- 172.21.15.129:0/1153877243 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1073a0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 --2- 172.21.15.129:0/1153877243 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 0x7f026c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 -- 172.21.15.129:0/1153877243 >> 172.21.15.129:0/1153877243 conn(0x7f026c0fc1c0 msgr2=0x7f026c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 -- 172.21.15.129:0/1153877243 shutdown_connections 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.633+0000 7f02716af640 1 -- 172.21.15.129:0/1153877243 wait complete. 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 Processor -- start 2024-09-05T01:28:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 -- start start 2024-09-05T01:28:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 0x7f026c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f026c10d680 con 0x7f026c104fb0 2024-09-05T01:28:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f026c10d500 con 0x7f026c1078e0 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02716af640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f026c10d800 con 0x7f026c102680 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40036/0 (socket says 172.21.15.129:40036) 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.634+0000 7f02627fc640 1 -- 172.21.15.129:0/317102720 learned_addr learned my addr 172.21.15.129:0/317102720 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f026b7fe640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f026affd640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 0x7f026c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 -- 172.21.15.129:0/317102720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 msgr2=0x7f026c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 0x7f026c197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 -- 172.21.15.129:0/317102720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 msgr2=0x7f026c19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 -- 172.21.15.129:0/317102720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c19f980 con 0x7f026c104fb0 2024-09-05T01:28:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f026b7fe640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02627fc640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f02540314e0 tx=0x7f02540042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f0268ff9640 1 -- 172.21.15.129:0/317102720 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025403f030 con 0x7f026c104fb0 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f0268ff9640 1 -- 172.21.15.129:0/317102720 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0254003cb0 con 0x7f026c104fb0 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026c19fc10 con 0x7f026c104fb0 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f0268ff9640 1 -- 172.21.15.129:0/317102720 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254035070 con 0x7f026c104fb0 2024-09-05T01:28:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.635+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026c1a0100 con 0x7f026c104fb0 2024-09-05T01:28:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.637+0000 7f0268ff9640 1 -- 172.21.15.129:0/317102720 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f0254042030 con 0x7f026c104fb0 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.637+0000 7f0274440740 1 Processor -- start 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 -- start start 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b539a0 0x55fce0b53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b58da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b5b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fce0ab72f0 con 0x55fce0b59370 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fce0a5f270 con 0x55fce0b542e0 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0274440740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fce0ab32b0 con 0x55fce0b539a0 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b5b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b5b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40046/0 (socket says 172.21.15.129:40046) 2024-09-05T01:28:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f02627fc640 1 -- 172.21.15.129:0/1292460018 learned_addr learned my addr 172.21.15.129:0/1292460018 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b58da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0268ff9640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 0x7f02580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026b7fe640 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b539a0 0x55fce0b53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026b7fe640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 0x7f02580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f0268ff9640 1 -- 172.21.15.129:0/317102720 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f02540efb80 con 0x7f026c104fb0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 -- 172.21.15.129:0/1292460018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b539a0 msgr2=0x55fce0b53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b539a0 0x55fce0b53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 -- 172.21.15.129:0/1292460018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 msgr2=0x55fce0b5b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b5b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.638+0000 7f026affd640 1 -- 172.21.15.129:0/1292460018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce0a5e340 con 0x55fce0b542e0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.639+0000 7f026affd640 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b58da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f025c009aa0 tx=0x7f025c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6518c4f1e5f23240 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.639+0000 7f026b7fe640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 0x7f02580a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0264009270 tx=0x7f0264005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.639+0000 7f02617fa640 1 -- 172.21.15.129:0/1292460018 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c016070 con 0x55fce0b542e0 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.639+0000 7f02617fa640 1 -- 172.21.15.129:0/1292460018 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c0054d0 con 0x55fce0b542e0 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.639+0000 7f02617fa640 1 -- 172.21.15.129:0/1292460018 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c004de0 con 0x55fce0b542e0 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- 172.21.15.129:0/1292460018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 msgr2=0x55fce0b58da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b58da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f025c009aa0 tx=0x7f025c009f60 comp rx=0 tx=0).stop 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- 172.21.15.129:0/1292460018 shutdown_connections 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b5b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b58da0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- 172.21.15.129:0/1292460018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b539a0 0x55fce0b53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- 172.21.15.129:0/1292460018 >> 172.21.15.129:0/1292460018 conn(0x55fce0b4f010 msgr2=0x55fce0b4f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- 172.21.15.129:0/1292460018 shutdown_connections 2024-09-05T01:28:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- 172.21.15.129:0/1292460018 wait complete. 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- start start 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 Processor -- start 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b5db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b542e0 0x55fce0b5e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fce0b5c340 con 0x55fce0b59370 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fce0a40290 con 0x55fce0b539a0 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fce0b5c4c0 con 0x55fce0b542e0 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f0274440740 10 client.0 _ll_register_callbacks cb 0x55fce0a71140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.640+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40054/0 (socket says 172.21.15.129:40054) 2024-09-05T01:28:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 -- 172.21.15.129:0/28840504 learned_addr learned my addr 172.21.15.129:0/28840504 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026affd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b5db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02627fc640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b542e0 0x55fce0b5e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 -- 172.21.15.129:0/28840504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b542e0 msgr2=0x55fce0b5e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b542e0 0x55fce0b5e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 -- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 msgr2=0x55fce0b5db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b5db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 -- 172.21.15.129:0/28840504 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce07ca210 con 0x55fce0b59370 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f026b7fe640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0264014370 tx=0x7f0264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1cf38bf1027cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02527fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264024030 con 0x55fce0b59370 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f0264012970 con 0x55fce0b59370 2024-09-05T01:28:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f025c0054d0 con 0x55fce0b59370 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0264031050 con 0x55fce0b59370 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f0274440740 10 client.16680 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fce0896620 con 0x55fce0b59370 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.641+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c0054d0 con 0x55fce0b59370 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f026401d050 con 0x55fce0b59370 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 10 client.16680 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 20 client.16680 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0b6bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 4 client.16680 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fce0ac4a20 con 0x55fce0b698a0 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f026affd640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0b6bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f026affd640 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0b6bcf0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22157a36d7bfdd83 server_cookie=1ea21efcd0b8cd35 in_seq=0 out_seq=0 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.642+0000 7f02527fc640 10 client.16680 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f02527fc640 1 -- 172.21.15.129:0/28840504 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fce0ac4a20 con 0x55fce0b698a0 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f02527fc640 10 client.16680 handle_command_reply: tid=0 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 client.16680 shutdown 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 msgr2=0x55fce0b650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0264014370 tx=0x7f0264004a40 comp rx=0 tx=0).stop 2024-09-05T01:28:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 shutdown_connections 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0b6bcf0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b59370 0x55fce0b650e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b542e0 0x55fce0b5e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 --2- 172.21.15.129:0/28840504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b5db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f02527fc640 0 client.16680 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 >> 172.21.15.129:0/28840504 conn(0x55fce0b4f010 msgr2=0x55fce0b50820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 shutdown_connections 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 1 -- 172.21.15.129:0/28840504 wait complete. 2024-09-05T01:28:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.643+0000 7f0274440740 20 client.16680 trim_cache size 0 max 16384 2024-09-05T01:28:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.659+0000 7f0274440740 1 Processor -- start 2024-09-05T01:28:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.659+0000 7f0274440740 1 -- start start 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b4fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b50400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1ad130 con 0x55fce0b542e0 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1aaa00 con 0x55fce0b539a0 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0274440740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1a8850 con 0x55fce0b59370 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40068/0 (socket says 172.21.15.129:40068) 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 -- 172.21.15.129:0/3209023380 learned_addr learned my addr 172.21.15.129:0/3209023380 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f026affd640 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b4fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f026b7fe640 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b50400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 -- 172.21.15.129:0/3209023380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 msgr2=0x55fce0b50400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b50400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 -- 172.21.15.129:0/3209023380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 msgr2=0x55fce0b4fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b4fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 -- 172.21.15.129:0/3209023380 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fce0b590d0 con 0x55fce0b542e0 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f02627fc640 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f02540bbc70 tx=0x7f02540dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bcd59b2be4b6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0260ff9640 1 -- 172.21.15.129:0/3209023380 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254049040 con 0x55fce0b542e0 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0260ff9640 1 -- 172.21.15.129:0/3209023380 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02540f7040 con 0x55fce0b542e0 2024-09-05T01:28:06.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.660+0000 7f0260ff9640 1 -- 172.21.15.129:0/3209023380 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02540c0320 con 0x55fce0b542e0 2024-09-05T01:28:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 -- 172.21.15.129:0/3209023380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 msgr2=0x55fce0b65000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f02540bbc70 tx=0x7f02540dc040 comp rx=0 tx=0).stop 2024-09-05T01:28:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 -- 172.21.15.129:0/3209023380 shutdown_connections 2024-09-05T01:28:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b50400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b542e0 0x55fce0b65000 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 --2- 172.21.15.129:0/3209023380 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b539a0 0x55fce0b4fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 -- 172.21.15.129:0/3209023380 >> 172.21.15.129:0/3209023380 conn(0x55fce0b4f010 msgr2=0x55fce0b5db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 -- 172.21.15.129:0/3209023380 shutdown_connections 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.661+0000 7f0274440740 1 -- 172.21.15.129:0/3209023380 wait complete. 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 -- start start 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 Processor -- start 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b62950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1a8ec0 con 0x55fce0b539a0 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1aa330 con 0x55fce0b542e0 2024-09-05T01:28:06.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f026c1a9590 con 0x55fce0b59370 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f0274440740 10 client.0 _ll_register_callbacks cb 0x7f0264014de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40074/0 (socket says 172.21.15.129:40074) 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 -- 172.21.15.129:0/950610737 learned_addr learned my addr 172.21.15.129:0/950610737 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f02627fc640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b62950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026b7fe640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 -- 172.21.15.129:0/950610737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 msgr2=0x55fce0b63960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b63960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 -- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 msgr2=0x55fce0b62950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b62950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.662+0000 7f026affd640 1 -- 172.21.15.129:0/950610737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fce0b63f70 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f026affd640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f025c005650 tx=0x7f025c00ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be4dc7aacfd1802 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c016040 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f025c031680 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02540f7040 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f025c03a050 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 10 client.16686 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fce0b652f0 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02540f7040 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f025c00f040 con 0x55fce0b539a0 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 10 client.16686 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 20 client.16686 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:06.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0bd6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 4 client.16686 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:28:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.663+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fce0abed70 con 0x55fce0b698a0 2024-09-05T01:28:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.664+0000 7f02627fc640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0bd6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.664+0000 7f02627fc640 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0bd6c30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0e4d39c3aac41 server_cookie=729aa342423ff686 in_seq=0 out_seq=0 2024-09-05T01:28:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.664+0000 7f02527fc640 10 client.16686 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f02527fc640 1 -- 172.21.15.129:0/950610737 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fce0abed70 con 0x55fce0b698a0 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f02527fc640 10 client.16686 handle_command_reply: tid=0 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 client.16686 shutdown 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 msgr2=0x55fce0b62410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f025c005650 tx=0x7f025c00ea80 comp rx=0 tx=0).stop 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 shutdown_connections 2024-09-05T01:28:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fce0b698a0 0x55fce0bd6c30 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fce0b59370 0x55fce0b63960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fce0b542e0 0x55fce0b62950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f02527fc640 0 client.16686 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 --2- 172.21.15.129:0/950610737 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fce0b539a0 0x55fce0b62410 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 >> 172.21.15.129:0/950610737 conn(0x55fce0b4f010 msgr2=0x55fce0b4d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 shutdown_connections 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 1 -- 172.21.15.129:0/950610737 wait complete. 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.680+0000 7f0274440740 20 client.16686 trim_cache size 0 max 16384 2024-09-05T01:28:06.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:28:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:28:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:28:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:28:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 msgr2=0x7f02580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 0x7f02580a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f0264009270 tx=0x7f0264005bd0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 msgr2=0x7f026c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f02540314e0 tx=0x7f02540042e0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 shutdown_connections 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f02580a2c30 0x7f02580a50f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f026c1078e0 0x7f026c19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f026c104fb0 0x7f026c1982a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 --2- 172.21.15.129:0/317102720 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f026c102680 0x7f026c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 >> 172.21.15.129:0/317102720 conn(0x7f026c0fc1c0 msgr2=0x7f026c1030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 shutdown_connections 2024-09-05T01:28:06.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:06.681+0000 7f02716af640 1 -- 172.21.15.129:0/317102720 wait complete. 2024-09-05T01:28:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:08 smithi183 ceph-mon[34589]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 232 KiB/s wr, 43 op/s 2024-09-05T01:28:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:08 smithi129 ceph-mon[30979]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 232 KiB/s wr, 43 op/s 2024-09-05T01:28:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:08 smithi194 ceph-mon[34941]: pgmap v738: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 232 KiB/s wr, 43 op/s 2024-09-05T01:28:10.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 665.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:28:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:10 smithi183 ceph-mon[34589]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 224 KiB/s wr, 36 op/s 2024-09-05T01:28:10.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:10 smithi129 ceph-mon[30979]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 224 KiB/s wr, 36 op/s 2024-09-05T01:28:10.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:10 smithi194 ceph-mon[34941]: pgmap v739: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 224 KiB/s wr, 36 op/s 2024-09-05T01:28:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:12 smithi183 ceph-mon[34589]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 233 KiB/s wr, 38 op/s 2024-09-05T01:28:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:12 smithi129 ceph-mon[30979]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 233 KiB/s wr, 38 op/s 2024-09-05T01:28:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:12 smithi194 ceph-mon[34941]: pgmap v740: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 233 KiB/s wr, 38 op/s 2024-09-05T01:28:12.929 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:28:13.191 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- 172.21.15.129:0/3543578925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258102ef0 msgr2=0x7f525810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 --2- 172.21.15.129:0/3543578925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258102ef0 0x7f525810da40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5248009ca0 tx=0x7f524802a950 comp rx=0 tx=0).stop 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- 172.21.15.129:0/3543578925 shutdown_connections 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 --2- 172.21.15.129:0/3543578925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258102ef0 0x7f525810da40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 --2- 172.21.15.129:0/3543578925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102570 0x7f52581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 --2- 172.21.15.129:0/3543578925 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258100e20 0x7f5258101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- 172.21.15.129:0/3543578925 >> 172.21.15.129:0/3543578925 conn(0x7f52580fc7d0 msgr2=0x7f52580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:13.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- 172.21.15.129:0/3543578925 shutdown_connections 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- 172.21.15.129:0/3543578925 wait complete. 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 Processor -- start 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.692+0000 7f525f586640 1 -- start start 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258102570 0x7f5258199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102ef0 0x7f52581a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5258110e30 con 0x7f5258100e20 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5258110cb0 con 0x7f5258102ef0 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525f586640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5258110fb0 con 0x7f5258102570 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:13.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51034/0 (socket says 172.21.15.129:51034) 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 -- 172.21.15.129:0/30225092 learned_addr learned my addr 172.21.15.129:0/30225092 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525cb21640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258102570 0x7f5258199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525db23640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102ef0 0x7f52581a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 -- 172.21.15.129:0/30225092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258102570 msgr2=0x7f5258199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258102570 0x7f5258199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 -- 172.21.15.129:0/30225092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102ef0 msgr2=0x7f52581a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102ef0 0x7f52581a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.693+0000 7f525d322640 1 -- 172.21.15.129:0/30225092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52581a47d0 con 0x7f5258100e20 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f525d322640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f525000c970 tx=0x7f525000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250007bb0 con 0x7f5258100e20 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5250007d50 con 0x7f5258100e20 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52581a49a0 con 0x7f5258100e20 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525000f450 con 0x7f5258100e20 2024-09-05T01:28:13.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.694+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52581a4e60 con 0x7f5258100e20 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.695+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5258101200 con 0x7f5258100e20 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.696+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5250004f20 con 0x7f5258100e20 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.696+0000 7f52567fc640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 0x7f52400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.697+0000 7f525cb21640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 0x7f52400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.697+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f52500cf1e0 con 0x7f5258100e20 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.697+0000 7f525cb21640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 0x7f52400a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f525806aac0 tx=0x7f524c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:13.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.701+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5250014030 con 0x7f5258100e20 2024-09-05T01:28:13.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.912+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f525806a560 con 0x7f5258100e20 2024-09-05T01:28:13.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.913+0000 7f52567fc640 1 -- 172.21.15.129:0/30225092 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f525806a560 con 0x7f5258100e20 2024-09-05T01:28:13.914 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:28:13.914 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:28:13.914 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:28:13.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 msgr2=0x7f52400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:13.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 0x7f52400a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f525806aac0 tx=0x7f524c00b390 comp rx=0 tx=0).stop 2024-09-05T01:28:13.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 msgr2=0x7f5258199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f525000c970 tx=0x7f525000ce30 comp rx=0 tx=0).stop 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 shutdown_connections 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f52400a2b80 0x7f52400a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5258102ef0 0x7f52581a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5258102570 0x7f5258199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 --2- 172.21.15.129:0/30225092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5258100e20 0x7f5258199210 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 >> 172.21.15.129:0/30225092 conn(0x7f52580fc7d0 msgr2=0x7f52580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 shutdown_connections 2024-09-05T01:28:13.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:13.915+0000 7f525f586640 1 -- 172.21.15.129:0/30225092 wait complete. 2024-09-05T01:28:14.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:13 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/30225092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:28:14.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:13 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/30225092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:28:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:13 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/30225092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:28:14.738 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id eeee04fa --release --await' 2024-09-05T01:28:14.738 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id eeee04fa --release --await 2024-09-05T01:28:14.992 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:28:15.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 670.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:28:15.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:14 smithi129 ceph-mon[30979]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 219 KiB/s wr, 36 op/s 2024-09-05T01:28:15.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:14 smithi194 ceph-mon[34941]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 219 KiB/s wr, 36 op/s 2024-09-05T01:28:15.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:14 smithi183 ceph-mon[34589]: pgmap v741: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 219 KiB/s wr, 36 op/s 2024-09-05T01:28:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:16 smithi183 ceph-mon[34589]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 KiB/s wr, 34 op/s 2024-09-05T01:28:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:16 smithi129 ceph-mon[30979]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 KiB/s wr, 34 op/s 2024-09-05T01:28:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:16 smithi194 ceph-mon[34941]: pgmap v742: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 213 KiB/s wr, 34 op/s 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- 172.21.15.129:0/1131536412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 msgr2=0x7f1a68106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1131536412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68106840 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009930 tx=0x7f1a5002a420 comp rx=0 tx=0).stop 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- 172.21.15.129:0/1131536412 shutdown_connections 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1131536412 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a68106f50 0x7f1a6810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1131536412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68106840 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1131536412 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a681002f0 0x7f1a681006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- 172.21.15.129:0/1131536412 >> 172.21.15.129:0/1131536412 conn(0x7f1a68077210 msgr2=0x7f1a680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- 172.21.15.129:0/1131536412 shutdown_connections 2024-09-05T01:28:16.955 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- 172.21.15.129:0/1131536412 wait complete. 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 Processor -- start 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.270+0000 7f1a6df43640 1 -- start start 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 0x7f1a68198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a68106f50 0x7f1a6819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68111e90 con 0x7f1a68100ca0 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68111d10 con 0x7f1a68106f50 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68112010 con 0x7f1a681002f0 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51058/0 (socket says 172.21.15.129:51058) 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 -- 172.21.15.129:0/1258849233 learned_addr learned my addr 172.21.15.129:0/1258849233 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:16.956 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a67fff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a68106f50 0x7f1a6819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a677fe640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 0x7f1a68198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 -- 172.21.15.129:0/1258849233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 msgr2=0x7f1a68198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 0x7f1a68198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 -- 172.21.15.129:0/1258849233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a68106f50 msgr2=0x7f1a6819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a68106f50 0x7f1a6819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 -- 172.21.15.129:0/1258849233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6819d400 con 0x7f1a68100ca0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a677fe640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 0x7f1a68198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009230 tx=0x7f1a50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a657fa640 1 -- 172.21.15.129:0/1258849233 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5003f030 con 0x7f1a68100ca0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6819d690 con 0x7f1a68100ca0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a657fa640 1 -- 172.21.15.129:0/1258849233 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a50003aa0 con 0x7f1a68100ca0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.271+0000 7f1a657fa640 1 -- 172.21.15.129:0/1258849233 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50035070 con 0x7f1a68100ca0 2024-09-05T01:28:16.957 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.272+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681a4fd0 con 0x7f1a68100ca0 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.274+0000 7f1a657fa640 1 -- 172.21.15.129:0/1258849233 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1a50042030 con 0x7f1a68100ca0 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.274+0000 7f1a7031c740 1 Processor -- start 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.274+0000 7f1a7031c740 1 -- start start 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6127440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6132240 0x5582b6134630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5582b608c620 con 0x5582b6127980 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582b6033e50 con 0x5582b6132240 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a7031c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5582b60885e0 con 0x5582b6127060 2024-09-05T01:28:16.958 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a657fa640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 0x7f1a540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51074/0 (socket says 172.21.15.129:51074) 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 -- 172.21.15.129:0/3837340631 learned_addr learned my addr 172.21.15.129:0/3837340631 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a67fff640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6127440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6132240 0x5582b6134630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a67fff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 0x7f1a540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a657fa640 1 -- 172.21.15.129:0/1258849233 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a500ef9d0 con 0x7f1a68100ca0 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 -- 172.21.15.129:0/3837340631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 msgr2=0x5582b6127440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6127440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 -- 172.21.15.129:0/3837340631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6132240 msgr2=0x5582b6134630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6132240 0x5582b6134630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.959 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 -- 172.21.15.129:0/3837340631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b6024d40 con 0x5582b6127980 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a67fff640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6127440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a67fff640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 0x7f1a540a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a60009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.275+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cca0 tx=0x7f1a5800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb1643c8794bcb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a5dffb640 1 -- 172.21.15.129:0/3837340631 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58005220 con 0x5582b6127980 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a5dffb640 1 -- 172.21.15.129:0/3837340631 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a58004c50 con 0x5582b6127980 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a5dffb640 1 -- 172.21.15.129:0/3837340631 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a58010790 con 0x5582b6127980 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 -- 172.21.15.129:0/3837340631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 msgr2=0x5582b6131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.960 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cca0 tx=0x7f1a5800e720 comp rx=0 tx=0).stop 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 -- 172.21.15.129:0/3837340631 shutdown_connections 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6132240 0x5582b6134630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127980 0x5582b6131d00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3837340631 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6127440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 -- 172.21.15.129:0/3837340631 >> 172.21.15.129:0/3837340631 conn(0x5582b61235f0 msgr2=0x5582b61239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 -- 172.21.15.129:0/3837340631 shutdown_connections 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.276+0000 7f1a7031c740 1 -- 172.21.15.129:0/3837340631 wait complete. 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 -- start start 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 Processor -- start 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127060 0x5582b612f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127980 0x5582b612f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.961 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5582b6134e10 con 0x5582b6132240 2024-09-05T01:28:16.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5582b6014f10 con 0x5582b6127060 2024-09-05T01:28:16.962 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5582b6134f90 con 0x5582b6127980 2024-09-05T01:28:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a7031c740 10 client.0 _ll_register_callbacks cb 0x5582b6038db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.963 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51084/0 (socket says 172.21.15.129:51084) 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.277+0000 7f1a67fff640 1 -- 172.21.15.129:0/3943694434 learned_addr learned my addr 172.21.15.129:0/3943694434 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127060 0x5582b612f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127980 0x5582b612f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 -- 172.21.15.129:0/3943694434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127980 msgr2=0x5582b612f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127980 0x5582b612f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 -- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127060 msgr2=0x5582b612f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127060 0x5582b612f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 -- 172.21.15.129:0/3943694434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b5d9b580 con 0x5582b6132240 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a67fff640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1a6001a700 tx=0x7f1a60004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cc46c8e149904 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a6002a060 con 0x5582b6132240 2024-09-05T01:28:16.964 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a6001acf0 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a58004c50 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1a60031050 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a7031c740 10 client.16704 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5582b5e65960 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.278+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a58004c50 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1a600183a0 con 0x5582b6132240 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 10 client.16704 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 10 client.16704 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 20 client.16704 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b6141d10 0x5582b6144160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 4 client.16704 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582b6099b90 con 0x5582b6141d10 2024-09-05T01:28:16.965 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b6141d10 0x5582b6144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a46ffd640 10 client.16704 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.279+0000 7f1a677fe640 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b6141d10 0x5582b6144160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42911a6ffe7a3f41 server_cookie=37b22fc747f5a299 in_seq=0 out_seq=0 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a46ffd640 1 -- 172.21.15.129:0/3943694434 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582b6099b90 con 0x5582b6141d10 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a46ffd640 10 client.16704 handle_command_reply: tid=0 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 client.16704 shutdown 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 msgr2=0x5582b612fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1a6001a700 tx=0x7f1a60004b50 comp rx=0 tx=0).stop 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 shutdown_connections 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b6141d10 0x5582b6144160 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b612fb60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127980 0x5582b612f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.966 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 --2- 172.21.15.129:0/3943694434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127060 0x5582b612f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a46ffd640 0 client.16704 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.280+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 >> 172.21.15.129:0/3943694434 conn(0x5582b61235f0 msgr2=0x5582b61292d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.281+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 shutdown_connections 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.281+0000 7f1a7031c740 1 -- 172.21.15.129:0/3943694434 wait complete. 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.281+0000 7f1a7031c740 20 client.16704 trim_cache size 0 max 16384 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.297+0000 7f1a7031c740 1 Processor -- start 2024-09-05T01:28:16.967 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 -- start start 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6130fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b6129ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681b17e0 con 0x5582b6132240 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681af0b0 con 0x5582b6127980 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a7031c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681acf00 con 0x5582b6127060 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51088/0 (socket says 172.21.15.129:51088) 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 -- 172.21.15.129:0/2902633498 learned_addr learned my addr 172.21.15.129:0/2902633498 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b6129ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a677fe640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6130fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 -- 172.21.15.129:0/2902633498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 msgr2=0x5582b6130fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6130fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 -- 172.21.15.129:0/2902633498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 msgr2=0x5582b6129ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b6129ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 -- 172.21.15.129:0/2902633498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582b61277b0 con 0x5582b6132240 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a677fe640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6130fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a67fff640 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1a600156b0 tx=0x7f1a600049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1be65938f073ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a5d7fa640 1 -- 172.21.15.129:0/2902633498 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60014040 con 0x5582b6132240 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a5d7fa640 1 -- 172.21.15.129:0/2902633498 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a6000b7f0 con 0x5582b6132240 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.298+0000 7f1a5d7fa640 1 -- 172.21.15.129:0/2902633498 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a60018bc0 con 0x5582b6132240 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- 172.21.15.129:0/2902633498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 msgr2=0x5582b613a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1a600156b0 tx=0x7f1a600049c0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- 172.21.15.129:0/2902633498 shutdown_connections 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6132240 0x5582b613a5c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.969 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b6129ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 --2- 172.21.15.129:0/2902633498 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6127060 0x5582b6130fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- 172.21.15.129:0/2902633498 >> 172.21.15.129:0/2902633498 conn(0x5582b61235f0 msgr2=0x5582b6122750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- 172.21.15.129:0/2902633498 shutdown_connections 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- 172.21.15.129:0/2902633498 wait complete. 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.299+0000 7f1a7031c740 1 -- start start 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 Processor -- start 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b61a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6132240 0x5582b61a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681ad570 con 0x5582b6127060 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681ae9e0 con 0x5582b6127980 2024-09-05T01:28:16.970 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681adc40 con 0x5582b6132240 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a7031c740 10 client.0 _ll_register_callbacks cb 0x7f1a60015ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51096/0 (socket says 172.21.15.129:51096) 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 -- 172.21.15.129:0/571438440 learned_addr learned my addr 172.21.15.129:0/571438440 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b61a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a67fff640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6132240 0x5582b61a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 -- 172.21.15.129:0/571438440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6132240 msgr2=0x5582b61a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6132240 0x5582b61a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 -- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 msgr2=0x5582b61a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b61a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.300+0000 7f1a677fe640 1 -- 172.21.15.129:0/571438440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582b61aa0b0 con 0x5582b6127060 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a677fe640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cca0 tx=0x7f1a5800cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37210b6bd8664d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:16.971 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5802b070 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1a580108b0 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a6000b7f0 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1a58039050 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 10 client.16710 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5582b61b14a0 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a6000b7f0 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1a58030990 con 0x5582b6127060 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 10 client.16710 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 10 client.16710 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 20 client.16710 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b61b18f0 0x5582b61b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:16.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 4 client.16710 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eeee04fa", "release": true, "await": true}] 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.301+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eeee04fa", "release": true, "await": true}) -- 0x5582b60940a0 con 0x5582b61b18f0 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.302+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b61b18f0 0x5582b61b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 48, 2024-09-05T01:28:16.973 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "eeee04fa": { 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 48, 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 146.4, 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:28:16.974 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.302+0000 7f1a5ffff640 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b61b18f0 0x5582b61b3d10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e70e8a5f3b28fa3 server_cookie=708e46bdd755689f in_seq=0 out_seq=0 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.302+0000 7f1a46ffd640 10 client.16710 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:16.975 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a46ffd640 1 -- 172.21.15.129:0/571438440 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5582b60940a0 con 0x5582b61b18f0 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a46ffd640 10 client.16710 handle_command_reply: tid=0 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 client.16710 shutdown 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 msgr2=0x5582b61a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cca0 tx=0x7f1a5800cfa0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 shutdown_connections 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5582b61b18f0 0x5582b61b3d10 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5582b6132240 0x5582b61a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a46ffd640 0 client.16710 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5582b6127980 0x5582b61a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 --2- 172.21.15.129:0/571438440 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5582b6127060 0x5582b61a4ba0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.357+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 >> 172.21.15.129:0/571438440 conn(0x5582b61235f0 msgr2=0x5582b6121f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.358+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 shutdown_connections 2024-09-05T01:28:16.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.358+0000 7f1a7031c740 1 -- 172.21.15.129:0/571438440 wait complete. 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.358+0000 7f1a7031c740 20 client.16710 trim_cache size 0 max 16384 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 msgr2=0x7f1a540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 0x7f1a540a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a60009650 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 msgr2=0x7f1a68198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009230 tx=0x7f1a50004270 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 shutdown_connections 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1a540a2b80 0x7f1a540a5040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a68106f50 0x7f1a6819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a68100ca0 0x7f1a68198990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 --2- 172.21.15.129:0/1258849233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1a681002f0 0x7f1a68198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 >> 172.21.15.129:0/1258849233 conn(0x7f1a68077210 msgr2=0x7f1a681052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:16.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 shutdown_connections 2024-09-05T01:28:16.978 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:28:16.359+0000 7f1a6df43640 1 -- 172.21.15.129:0/1258849233 wait complete. 2024-09-05T01:28:17.822 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eeee04fa', total seconds elapsed: 31.3 2024-09-05T01:28:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:18 smithi183 ceph-mon[34589]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 775 KiB/s wr, 73 op/s 2024-09-05T01:28:18.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:18 smithi129 ceph-mon[30979]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 775 KiB/s wr, 73 op/s 2024-09-05T01:28:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:18 smithi194 ceph-mon[34941]: pgmap v743: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 775 KiB/s wr, 73 op/s 2024-09-05T01:28:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 -- 172.21.15.129:0/2701272750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c1037a0 msgr2=0x7f8f9c10a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 --2- 172.21.15.129:0/2701272750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c10a030 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009900 tx=0x7f8f9402a410 comp rx=0 tx=0).stop 2024-09-05T01:28:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 -- 172.21.15.129:0/2701272750 shutdown_connections 2024-09-05T01:28:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 --2- 172.21.15.129:0/2701272750 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c10a030 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 --2- 172.21.15.129:0/2701272750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c102de0 0x7f8f9c103260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 --2- 172.21.15.129:0/2701272750 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.826+0000 7f8fa4768640 1 -- 172.21.15.129:0/2701272750 >> 172.21.15.129:0/2701272750 conn(0x7f8f9c0fb970 msgr2=0x7f8f9c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.827+0000 7f8fa4768640 1 -- 172.21.15.129:0/2701272750 shutdown_connections 2024-09-05T01:28:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.827+0000 7f8fa4768640 1 -- 172.21.15.129:0/2701272750 wait complete. 2024-09-05T01:28:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.828+0000 7f8fa4768640 1 Processor -- start 2024-09-05T01:28:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa4768640 1 -- start start 2024-09-05T01:28:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa4768640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa4768640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 0x7f8f9c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:40998/0 (socket says 172.21.15.129:40998) 2024-09-05T01:28:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 0x7f8f9c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 0x7f8f9c1981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55186/0 (socket says 172.21.15.129:55186) 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.829+0000 7f8fa4768640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c19f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 -- 172.21.15.129:0/1404097018 learned_addr learned my addr 172.21.15.129:0/1404097018 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10d670 con 0x7f8f9c1037a0 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10d4f0 con 0x7f8f9c102de0 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c10d7f0 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c19f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 -- 172.21.15.129:0/1404097018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 msgr2=0x7f8f9c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 0x7f8f9c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 -- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 msgr2=0x7f8f9c19f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c19f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c19f870 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.830+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800ebc0 tx=0x7f8f9800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.831+0000 7f8f937fe640 1 -- 172.21.15.129:0/1404097018 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98003b70 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.831+0000 7f8f937fe640 1 -- 172.21.15.129:0/1404097018 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f98003d10 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.831+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c19fb60 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.831+0000 7f8f937fe640 1 -- 172.21.15.129:0/1404097018 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98019820 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.831+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1a0100 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.835 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.834+0000 7f8f937fe640 1 -- 172.21.15.129:0/1404097018 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8f980199c0 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.835+0000 7f8f937fe640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 0x7f8f880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.836 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.836+0000 7f8f937fe640 1 -- 172.21.15.129:0/1404097018 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8f980cfde0 con 0x7f8f9c0ffe10 2024-09-05T01:28:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.836+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 0x7f8f880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.838+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 0x7f8f880a5240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00bbe0 tx=0x7f8f8c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.838+0000 7f8fa75d9740 1 Processor -- start 2024-09-05T01:28:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.838+0000 7f8fa75d9740 1 -- start start 2024-09-05T01:28:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bddd9a0 0x56550bdddda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 0x56550bded680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bdf0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56550bd432f0 con 0x56550bdedbc0 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56550bceb270 con 0x56550bdde2e0 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56550bd3f2b0 con 0x56550bddd9a0 2024-09-05T01:28:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 0x56550bded680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 0x56550bded680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55200/0 (socket says 172.21.15.129:55200) 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa1d03640 1 -- 172.21.15.129:0/1847028095 learned_addr learned my addr 172.21.15.129:0/1847028095 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.839+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bdf0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.840+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bddd9a0 0x56550bdddda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.840+0000 7f8fa2d05640 1 -- 172.21.15.129:0/1847028095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bddd9a0 msgr2=0x56550bdddda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.840+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bddd9a0 0x56550bdddda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.841+0000 7f8fa2d05640 1 -- 172.21.15.129:0/1847028095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 msgr2=0x56550bded680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.841+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 0x56550bded680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.841+0000 7f8fa2d05640 1 -- 172.21.15.129:0/1847028095 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550bcea340 con 0x56550bdedbc0 2024-09-05T01:28:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.841+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bdf0030 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8f94005dc0 tx=0x7f8f940040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842bb4df1232b894 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.842+0000 7f8f90ff9640 1 -- 172.21.15.129:0/1847028095 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9403f070 con 0x56550bdedbc0 2024-09-05T01:28:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.842+0000 7f8f90ff9640 1 -- 172.21.15.129:0/1847028095 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f94004440 con 0x56550bdedbc0 2024-09-05T01:28:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.843+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1847028095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 msgr2=0x56550bdf0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.843+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bdf0030 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8f94005dc0 tx=0x7f8f940040d0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.846+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1847028095 shutdown_connections 2024-09-05T01:28:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.846+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bdf0030 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.846+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bdde2e0 0x56550bded680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.846+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1847028095 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bddd9a0 0x56550bdddda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.846+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1847028095 >> 172.21.15.129:0/1847028095 conn(0x56550bddaf90 msgr2=0x56550bddb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.851+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1847028095 shutdown_connections 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1847028095 wait complete. 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 -- start start 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 Processor -- start 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be03930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.852+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56550be01b40 con 0x56550bdedbc0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56550bccc290 con 0x56550bddd9a0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa2d05640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa2d05640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51124/0 (socket says 172.21.15.129:51124) 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa2d05640 1 -- 172.21.15.129:0/3445214652 learned_addr learned my addr 172.21.15.129:0/3445214652 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56550be01cc0 con 0x56550bdde2e0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be03930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa75d9740 10 client.0 _ll_register_callbacks cb 0x56550bcfd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa2504640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 msgr2=0x56550be033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3445214652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 msgr2=0x56550be0a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.853+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3445214652 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550ba56210 con 0x56550bdde2e0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be03930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56550bddec70 tx=0x7f8f8c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40bad84ad2b1ee07 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01fb40 con 0x56550bdde2e0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8f8c01fce0 con 0x56550bdde2e0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.854+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f94004440 con 0x56550bdde2e0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.855+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f8c005020 con 0x56550bdde2e0 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.855+0000 7f8fa75d9740 10 client.26155 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56550bb22620 con 0x56550bdde2e0 2024-09-05T01:28:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f94004440 con 0x56550bdde2e0 2024-09-05T01:28:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8f8c026070 con 0x56550bdde2e0 2024-09-05T01:28:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8fa75d9740 10 client.26155 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8fa75d9740 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8fa75d9740 20 client.26155 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.856+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be0f0a0 0x56550be11510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.857+0000 7f8fa75d9740 4 client.26155 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:28:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.857+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56550bd50a20 con 0x56550be0f0a0 2024-09-05T01:28:19.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.857+0000 7f8fa2504640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be0f0a0 0x56550be11510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.860+0000 7f8fa2504640 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be0f0a0 0x56550be11510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20e384e8eec809bd server_cookie=d8cb079e0f997c89 in_seq=0 out_seq=0 2024-09-05T01:28:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.860+0000 7f8f85ffb640 10 client.26155 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.861+0000 7f8f85ffb640 1 -- 172.21.15.129:0/3445214652 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56550bd50a20 con 0x56550be0f0a0 2024-09-05T01:28:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.861+0000 7f8f85ffb640 10 client.26155 handle_command_reply: tid=0 2024-09-05T01:28:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 client.26155 shutdown 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 msgr2=0x56550be03930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be03930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x56550bddec70 tx=0x7f8f8c004730 comp rx=0 tx=0).stop 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 shutdown_connections 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be0f0a0 0x56550be11510 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550be0a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be03930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3445214652 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.862+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 >> 172.21.15.129:0/3445214652 conn(0x56550bddaf90 msgr2=0x56550bddc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.863+0000 7f8f85ffb640 0 client.26155 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.863+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 shutdown_connections 2024-09-05T01:28:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.863+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3445214652 wait complete. 2024-09-05T01:28:19.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.863+0000 7f8fa75d9740 20 client.26155 trim_cache size 0 max 16384 2024-09-05T01:28:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.890+0000 7f8fa75d9740 1 Processor -- start 2024-09-05T01:28:19.890 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.890+0000 7f8fa75d9740 1 -- start start 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.890+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.890+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.890+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bddbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55214/0 (socket says 172.21.15.129:55214) 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa1d03640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:41030/0 (socket says 172.21.15.129:41030) 2024-09-05T01:28:19.892 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa2504640 1 -- 172.21.15.129:0/3386656372 learned_addr learned my addr 172.21.15.129:0/3386656372 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c1a2690 con 0x56550bdedbc0 2024-09-05T01:28:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c074040 con 0x56550bddd9a0 2024-09-05T01:28:19.893 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c071e90 con 0x56550bdde2e0 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.891+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bddbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3386656372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 msgr2=0x56550bde9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3386656372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 msgr2=0x56550bddbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bddbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 -- 172.21.15.129:0/3386656372 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550bddea90 con 0x56550bdde2e0 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa2504640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c031510 tx=0x7f8f8c00b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c217778e1d0248b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bddbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8f87fff640 1 -- 172.21.15.129:0/3386656372 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01f240 con 0x56550bdde2e0 2024-09-05T01:28:19.894 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8f87fff640 1 -- 172.21.15.129:0/3386656372 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f8c01f3e0 con 0x56550bdde2e0 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.892+0000 7f8f87fff640 1 -- 172.21.15.129:0/3386656372 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c004b20 con 0x56550bdde2e0 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.893+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 msgr2=0x56550be0a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.893+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c031510 tx=0x7f8f8c00b590 comp rx=0 tx=0).stop 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.894+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 shutdown_connections 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.894+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdedbc0 0x56550bddbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.894+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdde2e0 0x56550be0a930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.894+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/3386656372 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550bde9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.895 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.894+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 >> 172.21.15.129:0/3386656372 conn(0x56550bddaf90 msgr2=0x56550bde9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.895+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 shutdown_connections 2024-09-05T01:28:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.895+0000 7f8fa75d9740 1 -- 172.21.15.129:0/3386656372 wait complete. 2024-09-05T01:28:19.896 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.896+0000 7f8fa75d9740 1 -- start start 2024-09-05T01:28:19.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.896+0000 7f8fa75d9740 1 Processor -- start 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdde2e0 0x56550be06240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 0x56550be091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c072500 con 0x56550bdde2e0 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c073970 con 0x56550bddd9a0 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c072bd0 con 0x56550bdedbc0 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa75d9740 10 client.0 _ll_register_callbacks cb 0x7f8f8c010db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:19.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa2504640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55230/0 (socket says 172.21.15.129:55230) 2024-09-05T01:28:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa2504640 1 -- 172.21.15.129:0/1851359279 learned_addr learned my addr 172.21.15.129:0/1851359279 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.897+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 0x56550be091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 -- 172.21.15.129:0/1851359279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 msgr2=0x56550be091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 0x56550be091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 -- 172.21.15.129:0/1851359279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdde2e0 msgr2=0x56550be06240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdde2e0 0x56550be06240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 -- 172.21.15.129:0/1851359279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56550be09800 con 0x56550bddd9a0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2d05640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 0x56550be091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa2504640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f980bc010 tx=0x7f8f980047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eeceed18eb49b35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f980a0580 con 0x56550bddd9a0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8f980bc0b0 con 0x56550bddd9a0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.898+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c01f3e0 con 0x56550bddd9a0 2024-09-05T01:28:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f980bc2b0 con 0x56550bddd9a0 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 10 client.26288 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56550be0aba0 con 0x56550bddd9a0 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f8c01f3e0 con 0x56550bddd9a0 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f8f980a49b0 con 0x56550bddd9a0 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 10 client.26288 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 20 client.26288 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be7a490 0x56550be7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.899+0000 7f8fa75d9740 4 client.26288 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.900+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56550bd4ad70 con 0x56550be7a490 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.900+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be7a490 0x56550be7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.900+0000 7f8f85ffb640 10 client.26288 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.900+0000 7f8fa1d03640 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be7a490 0x56550be7c8b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae32ab70c863218a server_cookie=268f59ea8073a823 in_seq=0 out_seq=0 2024-09-05T01:28:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.937+0000 7f8f85ffb640 1 -- 172.21.15.129:0/1851359279 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56550bd4ad70 con 0x56550be7a490 2024-09-05T01:28:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.937+0000 7f8f85ffb640 10 client.26288 handle_command_reply: tid=0 2024-09-05T01:28:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.937+0000 7f8fa75d9740 1 client.26288 shutdown 2024-09-05T01:28:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 msgr2=0x56550be05d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f980bc010 tx=0x7f8f980047e0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 shutdown_connections 2024-09-05T01:28:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56550be7a490 0x56550be7c8b0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.939 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8f85ffb640 0 client.26288 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56550bdedbc0 0x56550be091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56550bdde2e0 0x56550be06240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 --2- 172.21.15.129:0/1851359279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56550bddd9a0 0x56550be05d00 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8f980bc010 tx=0x7f8f980047e0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.938+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 >> 172.21.15.129:0/1851359279 conn(0x56550bddaf90 msgr2=0x56550bdd99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.939+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 shutdown_connections 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.939+0000 7f8fa75d9740 1 -- 172.21.15.129:0/1851359279 wait complete. 2024-09-05T01:28:19.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.939+0000 7f8fa75d9740 20 client.26288 trim_cache size 0 max 16384 2024-09-05T01:28:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:28:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:28:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:28:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:28:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.941+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 msgr2=0x7f8f880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.941+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 0x7f8f880a5240 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00bbe0 tx=0x7f8f8c00b390 comp rx=0 tx=0).stop 2024-09-05T01:28:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 msgr2=0x7f8f9c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800ebc0 tx=0x7f8f9800c6a0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 shutdown_connections 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8f880a2d80 0x7f8f880a5240 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f9c1037a0 0x7f8f9c19f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8f9c102de0 0x7f8f9c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 --2- 172.21.15.129:0/1404097018 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8f9c0ffe10 0x7f8f9c197c60 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800ebc0 tx=0x7f8f9800c6a0 comp rx=0 tx=0).stop 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 >> 172.21.15.129:0/1404097018 conn(0x7f8f9c0fb970 msgr2=0x7f8f9c106160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.942+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 shutdown_connections 2024-09-05T01:28:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:19.943+0000 7f8fa4768640 1 -- 172.21.15.129:0/1404097018 wait complete. 2024-09-05T01:28:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 675.0 s, 581.4 tps, lat 1005.745 ms stddev 4630.057 2024-09-05T01:28:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:20 smithi183 ceph-mon[34589]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-09-05T01:28:20.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:20 smithi129 ceph-mon[30979]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-09-05T01:28:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:20 smithi194 ceph-mon[34941]: pgmap v744: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.3 MiB/s wr, 110 op/s 2024-09-05T01:28:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:28:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:28:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:28:21.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:28:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:28:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:28:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:28:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:28:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:28:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:28:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:28:21.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:28:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:22 smithi183 ceph-mon[34589]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-05T01:28:22.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:22 smithi129 ceph-mon[30979]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-05T01:28:22.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:22 smithi194 ceph-mon[34941]: pgmap v745: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.3 MiB/s wr, 234 op/s 2024-09-05T01:28:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:24 smithi183 ceph-mon[34589]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-09-05T01:28:24.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:24 smithi129 ceph-mon[30979]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-09-05T01:28:24.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:24 smithi194 ceph-mon[34941]: pgmap v746: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 3.8 MiB/s wr, 389 op/s 2024-09-05T01:28:25.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 680.0 s, 776.4 tps, lat 128.623 ms stddev 63.090 2024-09-05T01:28:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:26 smithi183 ceph-mon[34589]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-05T01:28:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:26 smithi129 ceph-mon[30979]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-05T01:28:26.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:26 smithi194 ceph-mon[34941]: pgmap v747: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-05T01:28:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:28 smithi183 ceph-mon[34589]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-09-05T01:28:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:28 smithi129 ceph-mon[30979]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-09-05T01:28:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:28 smithi194 ceph-mon[34941]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 671 op/s 2024-09-05T01:28:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 685.0 s, 718.2 tps, lat 138.754 ms stddev 129.780 2024-09-05T01:28:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:30 smithi183 ceph-mon[34589]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 752 op/s 2024-09-05T01:28:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:30 smithi129 ceph-mon[30979]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 752 op/s 2024-09-05T01:28:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:30 smithi194 ceph-mon[34941]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.4 MiB/s wr, 752 op/s 2024-09-05T01:28:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:32 smithi183 ceph-mon[34589]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 838 op/s 2024-09-05T01:28:32.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:32 smithi129 ceph-mon[30979]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 838 op/s 2024-09-05T01:28:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:32 smithi194 ceph-mon[34941]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.9 MiB/s wr, 838 op/s 2024-09-05T01:28:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.092+0000 7f2f1f59e640 1 -- 172.21.15.129:0/3303487431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 msgr2=0x7f2f20100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.092+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/3303487431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f20100c40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009900 tx=0x7f2f1002a420 comp rx=0 tx=0).stop 2024-09-05T01:28:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.094+0000 7f2f1f59e640 1 -- 172.21.15.129:0/3303487431 shutdown_connections 2024-09-05T01:28:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.094+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/3303487431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f200698a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.094+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/3303487431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f20101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.094+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/3303487431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f20100c40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.094+0000 7f2f1f59e640 1 -- 172.21.15.129:0/3303487431 >> 172.21.15.129:0/3303487431 conn(0x7f2f200fbfa0 msgr2=0x7f2f200fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.097+0000 7f2f1f59e640 1 -- 172.21.15.129:0/3303487431 shutdown_connections 2024-09-05T01:28:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.097+0000 7f2f1f59e640 1 -- 172.21.15.129:0/3303487431 wait complete. 2024-09-05T01:28:33.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 Processor -- start 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 -- start start 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f2019e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f2019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f201a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010f700 con 0x7f2f20101a30 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010f580 con 0x7f2f20100840 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.098+0000 7f2f1f59e640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2010f880 con 0x7f2f201023f0 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.099+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f2019e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.099+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f2019e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59590/0 (socket says 172.21.15.129:59590) 2024-09-05T01:28:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.099+0000 7f2f1e59c640 1 -- 172.21.15.129:0/1338215226 learned_addr learned my addr 172.21.15.129:0/1338215226 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.099+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f201a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.099+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f2019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 -- 172.21.15.129:0/1338215226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 msgr2=0x7f2f2019e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f2019e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 -- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 msgr2=0x7f2f2019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f2019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 -- 172.21.15.129:0/1338215226 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f201a5ea0 con 0x7f2f201023f0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f2019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.100+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f201a5830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009fd0 tx=0x7f2f1800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.101+0000 7f2f0f7fe640 1 -- 172.21.15.129:0/1338215226 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1800ce20 con 0x7f2f201023f0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.101+0000 7f2f0f7fe640 1 -- 172.21.15.129:0/1338215226 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f18005020 con 0x7f2f201023f0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.101+0000 7f2f0f7fe640 1 -- 172.21.15.129:0/1338215226 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f18010690 con 0x7f2f201023f0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.101+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f201a6190 con 0x7f2f201023f0 2024-09-05T01:28:33.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.101+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a6710 con 0x7f2f201023f0 2024-09-05T01:28:33.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 Processor -- start 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 -- start start 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7de9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f0f7fe640 1 -- 172.21.15.129:0/1338215226 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2f180051d0 con 0x7f2f201023f0 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7def540 0x55f7b7df1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7d4d2f0 con 0x55f7b7def540 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7cf5270 con 0x55f7b7dea4b0 2024-09-05T01:28:33.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f27be9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7d492b0 con 0x55f7b7de9b70 2024-09-05T01:28:33.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59606/0 (socket says 172.21.15.129:59606) 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.106+0000 7f2f1e59c640 1 -- 172.21.15.129:0/181119637 learned_addr learned my addr 172.21.15.129:0/181119637 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.107+0000 7f2f1e59c640 1 -- 172.21.15.129:0/181119637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 msgr2=0x55f7b7de9f70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.107+0000 7f2f0f7fe640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 0x7f2f080a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.107+0000 7f2f0f7fe640 1 -- 172.21.15.129:0/1338215226 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2f180cfa80 con 0x7f2f201023f0 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7de9f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1e59c640 1 -- 172.21.15.129:0/181119637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7def540 msgr2=0x55f7b7df1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7def540 0x55f7b7df1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1e59c640 1 -- 172.21.15.129:0/181119637 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b7cf4340 con 0x55f7b7dea4b0 2024-09-05T01:28:33.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f100052a0 tx=0x7f2f100315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=978a5d091b9ce696 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 0x7f2f080a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f0cff9640 1 -- 172.21.15.129:0/181119637 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10035020 con 0x55f7b7dea4b0 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f0cff9640 1 -- 172.21.15.129:0/181119637 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10041070 con 0x55f7b7dea4b0 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.108+0000 7f2f0cff9640 1 -- 172.21.15.129:0/181119637 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f100046c0 con 0x55f7b7dea4b0 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.109+0000 7f2f27be9740 1 -- 172.21.15.129:0/181119637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 msgr2=0x55f7b7deef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.109+0000 7f2f27be9740 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2f100052a0 tx=0x7f2f100315b0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 -- 172.21.15.129:0/181119637 shutdown_connections 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7def540 0x55f7b7df1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7deef70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 --2- 172.21.15.129:0/181119637 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7de9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 -- 172.21.15.129:0/181119637 >> 172.21.15.129:0/181119637 conn(0x55f7b7de51e0 msgr2=0x55f7b7de55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:33.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.110+0000 7f2f27be9740 1 -- 172.21.15.129:0/181119637 shutdown_connections 2024-09-05T01:28:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.112+0000 7f2f1ed9d640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 0x7f2f080a5210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2f1801a040 tx=0x7f2f1800c890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.113+0000 7f2f27be9740 1 -- 172.21.15.129:0/181119637 wait complete. 2024-09-05T01:28:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.113+0000 7f2f27be9740 1 -- start start 2024-09-05T01:28:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.113+0000 7f2f27be9740 1 Processor -- start 2024-09-05T01:28:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7def540 0x55f7b7dfb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7df24b0 con 0x55f7b7dea4b0 2024-09-05T01:28:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7cd6290 con 0x55f7b7def540 2024-09-05T01:28:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f7b7df2630 con 0x55f7b7de9b70 2024-09-05T01:28:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f27be9740 10 client.0 _ll_register_callbacks cb 0x55f7b7d07140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.114+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:59636/0 (socket says 172.21.15.129:59636) 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/2442978542 learned_addr learned my addr 172.21.15.129:0/2442978542 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7def540 0x55f7b7dfb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7def540 msgr2=0x55f7b7dfb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7def540 0x55f7b7dfb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.115+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/2442978542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 msgr2=0x55f7b7df4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.116+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.116+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/2442978542 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b7a60210 con 0x55f7b7de9b70 2024-09-05T01:28:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.116+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f14015dc0 tx=0x7f2f14009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77d72d89c0bbbab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400ec70 con 0x55f7b7de9b70 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f10041070 con 0x55f7b7de9b70 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2f1400ee10 con 0x55f7b7de9b70 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f14026050 con 0x55f7b7de9b70 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f27be9740 10 client.26170 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.117+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f7b7b2c620 con 0x55f7b7de9b70 2024-09-05T01:28:33.120 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.118+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10041070 con 0x55f7b7de9b70 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2f14017440 con 0x55f7b7de9b70 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 10 client.26170 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 10 client.26170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 20 client.26170 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e10f60 0x55f7b7e13380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 4 client.26170 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7b7d5aa20 con 0x55f7b7e10f60 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.119+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e10f60 0x55f7b7e13380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.120+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e10f60 0x55f7b7e13380 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d13dffd035d6c93e server_cookie=f468cb0d4636673e in_seq=0 out_seq=0 2024-09-05T01:28:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.120+0000 7f2f05ffb640 10 client.26170 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.121+0000 7f2f05ffb640 1 -- 172.21.15.129:0/2442978542 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7b7d5aa20 con 0x55f7b7e10f60 2024-09-05T01:28:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.121+0000 7f2f05ffb640 10 client.26170 handle_command_reply: tid=0 2024-09-05T01:28:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.121+0000 7f2f27be9740 1 client.26170 shutdown 2024-09-05T01:28:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.121+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 msgr2=0x55f7b7df3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.121+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2f14015dc0 tx=0x7f2f14009850 comp rx=0 tx=0).stop 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 shutdown_connections 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e10f60 0x55f7b7e13380 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7def540 0x55f7b7dfb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 --2- 172.21.15.129:0/2442978542 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7de9b70 0x55f7b7df3e00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 >> 172.21.15.129:0/2442978542 conn(0x55f7b7de51e0 msgr2=0x55f7b7de69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.123+0000 7f2f05ffb640 0 client.26170 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.124+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 shutdown_connections 2024-09-05T01:28:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.124+0000 7f2f27be9740 1 -- 172.21.15.129:0/2442978542 wait complete. 2024-09-05T01:28:33.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.124+0000 7f2f27be9740 20 client.26170 trim_cache size 0 max 16384 2024-09-05T01:28:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.155+0000 7f2f27be9740 1 Processor -- start 2024-09-05T01:28:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 -- start start 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7de9b70 0x55f7b7df42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7ddfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201aaed0 con 0x55f7b7de9b70 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201a87a0 con 0x55f7b7dea4b0 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.156+0000 7f2f27be9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20074040 con 0x55f7b7def540 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59624/0 (socket says 172.21.15.129:59624) 2024-09-05T01:28:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 -- 172.21.15.129:0/38889662 learned_addr learned my addr 172.21.15.129:0/38889662 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7de9b70 0x55f7b7df42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 -- 172.21.15.129:0/38889662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 msgr2=0x55f7b7ddfed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7ddfed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 -- 172.21.15.129:0/38889662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7de9b70 msgr2=0x55f7b7df42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7de9b70 0x55f7b7df42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 -- 172.21.15.129:0/38889662 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7b7de5000 con 0x55f7b7dea4b0 2024-09-05T01:28:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.157+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f100052d0 tx=0x7f2f1002a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1a04e7fa09eba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.158+0000 7f2f07fff640 1 -- 172.21.15.129:0/38889662 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10035040 con 0x55f7b7dea4b0 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.158+0000 7f2f07fff640 1 -- 172.21.15.129:0/38889662 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10041040 con 0x55f7b7dea4b0 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.158+0000 7f2f07fff640 1 -- 172.21.15.129:0/38889662 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10050b10 con 0x55f7b7dea4b0 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 -- 172.21.15.129:0/38889662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 msgr2=0x55f7b7dfb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2f100052d0 tx=0x7f2f1002a9c0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 -- 172.21.15.129:0/38889662 shutdown_connections 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7ddfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7dea4b0 0x55f7b7dfb2a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 --2- 172.21.15.129:0/38889662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7de9b70 0x55f7b7df42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 -- 172.21.15.129:0/38889662 >> 172.21.15.129:0/38889662 conn(0x55f7b7de51e0 msgr2=0x55f7b7df3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 -- 172.21.15.129:0/38889662 shutdown_connections 2024-09-05T01:28:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.159+0000 7f2f27be9740 1 -- 172.21.15.129:0/38889662 wait complete. 2024-09-05T01:28:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.160+0000 7f2f27be9740 1 -- start start 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.160+0000 7f2f27be9740 1 Processor -- start 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7e79c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201a6c60 con 0x55f7b7dea4b0 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201a80d0 con 0x55f7b7de9b70 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201a7330 con 0x55f7b7def540 2024-09-05T01:28:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f27be9740 10 client.0 _ll_register_callbacks cb 0x7f2f14009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:59640/0 (socket says 172.21.15.129:59640) 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45542/0 (socket says 172.21.15.129:45542) 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/1637878812 learned_addr learned my addr 172.21.15.129:0/1637878812 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7e79c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/1637878812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 msgr2=0x55f7b7e79c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7e79c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/1637878812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 msgr2=0x55f7b7df8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1dd9b640 1 -- 172.21.15.129:0/1637878812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7b7e7a1c0 con 0x55f7b7de9b70 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.161+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.162+0000 7f2f1dd9b640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f140091c0 tx=0x7f2f140153e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e5e8e90ea4e884 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.162+0000 7f2f05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.162+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14017e10 con 0x55f7b7de9b70 2024-09-05T01:28:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.163+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2f14018450 con 0x55f7b7de9b70 2024-09-05T01:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.163+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f10041040 con 0x55f7b7de9b70 2024-09-05T01:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.163+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2f14018670 con 0x55f7b7de9b70 2024-09-05T01:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.163+0000 7f2f27be9740 10 client.26306 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.163+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f7b7e7b4b0 con 0x55f7b7de9b70 2024-09-05T01:28:33.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.164+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f10041040 con 0x55f7b7de9b70 2024-09-05T01:28:33.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f2f14005990 con 0x55f7b7de9b70 2024-09-05T01:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 10 client.26306 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 10 client.26306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 20 client.26306 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e7bd50 0x55f7b7e7e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 4 client.26306 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:28:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.166+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7b7d54d70 con 0x55f7b7e7bd50 2024-09-05T01:28:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.167+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e7bd50 0x55f7b7e7e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.168+0000 7f2f1e59c640 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e7bd50 0x55f7b7e7e170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e430a7f0dc1aea5 server_cookie=d8873741485fec07 in_seq=0 out_seq=0 2024-09-05T01:28:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:33.168+0000 7f2f05ffb640 10 client.26306 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:34.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:34 smithi183 ceph-mon[34589]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 864 op/s 2024-09-05T01:28:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:34 smithi129 ceph-mon[30979]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 864 op/s 2024-09-05T01:28:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:34 smithi194 ceph-mon[34941]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.4 MiB/s wr, 864 op/s 2024-09-05T01:28:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 690.0 s, 852.0 tps, lat 119.026 ms stddev 79.351 2024-09-05T01:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f05ffb640 1 -- 172.21.15.129:0/1637878812 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7b7d54d70 con 0x55f7b7e7bd50 2024-09-05T01:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f05ffb640 10 client.26306 handle_command_reply: tid=0 2024-09-05T01:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 client.26306 shutdown 2024-09-05T01:28:36.163 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 msgr2=0x55f7b7df81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f140091c0 tx=0x7f2f140153e0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 shutdown_connections 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f7b7e7bd50 0x55f7b7e7e170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f7b7def540 0x55f7b7e79c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f7b7dea4b0 0x55f7b7df8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 --2- 172.21.15.129:0/1637878812 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f7b7de9b70 0x55f7b7df81c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 >> 172.21.15.129:0/1637878812 conn(0x55f7b7de51e0 msgr2=0x55f7b7de6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f05ffb640 0 client.26306 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.162+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 shutdown_connections 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f27be9740 1 -- 172.21.15.129:0/1637878812 wait complete. 2024-09-05T01:28:36.164 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f27be9740 20 client.26306 trim_cache size 0 max 16384 2024-09-05T01:28:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:28:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:28:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:28:36.165 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:28:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 msgr2=0x7f2f080a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 0x7f2f080a5210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2f1801a040 tx=0x7f2f1800c890 comp rx=0 tx=0).stop 2024-09-05T01:28:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 msgr2=0x7f2f201a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:36.166 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.163+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f201a5830 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2f18009fd0 tx=0x7f2f1800ef10 comp rx=0 tx=0).stop 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 shutdown_connections 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2f080a2d50 0x7f2f080a5210 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2f201023f0 0x7f2f201a5830 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2f20101a30 0x7f2f2019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 --2- 172.21.15.129:0/1338215226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2f20100840 0x7f2f2019e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 >> 172.21.15.129:0/1338215226 conn(0x7f2f200fbfa0 msgr2=0x7f2f200fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 shutdown_connections 2024-09-05T01:28:36.167 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:36.164+0000 7f2f1f59e640 1 -- 172.21.15.129:0/1338215226 wait complete. 2024-09-05T01:28:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:36 smithi183 ceph-mon[34589]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-05T01:28:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:36 smithi129 ceph-mon[30979]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-05T01:28:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:36 smithi194 ceph-mon[34941]: pgmap v752: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.9 MiB/s wr, 788 op/s 2024-09-05T01:28:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:38 smithi183 ceph-mon[34589]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.6 MiB/s wr, 943 op/s 2024-09-05T01:28:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:38 smithi129 ceph-mon[30979]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.6 MiB/s wr, 943 op/s 2024-09-05T01:28:38.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:38 smithi194 ceph-mon[34941]: pgmap v753: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.6 MiB/s wr, 943 op/s 2024-09-05T01:28:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 695.0 s, 821.7 tps, lat 121.048 ms stddev 77.235 2024-09-05T01:28:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:40 smithi183 ceph-mon[34589]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 839 op/s 2024-09-05T01:28:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:40 smithi129 ceph-mon[30979]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 839 op/s 2024-09-05T01:28:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:40 smithi194 ceph-mon[34941]: pgmap v754: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 8.6 MiB/s wr, 839 op/s 2024-09-05T01:28:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:42 smithi183 ceph-mon[34589]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 874 op/s 2024-09-05T01:28:42.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:42 smithi129 ceph-mon[30979]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 874 op/s 2024-09-05T01:28:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:42 smithi194 ceph-mon[34941]: pgmap v755: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 8.9 MiB/s wr, 874 op/s 2024-09-05T01:28:44.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:44 smithi183 ceph-mon[34589]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.4 MiB/s wr, 903 op/s 2024-09-05T01:28:44.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:44 smithi129 ceph-mon[30979]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.4 MiB/s wr, 903 op/s 2024-09-05T01:28:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:44 smithi194 ceph-mon[34941]: pgmap v756: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.4 MiB/s wr, 903 op/s 2024-09-05T01:28:45.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 700.0 s, 891.4 tps, lat 112.511 ms stddev 76.689 2024-09-05T01:28:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:46 smithi183 ceph-mon[34589]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 826 op/s 2024-09-05T01:28:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:46 smithi129 ceph-mon[30979]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 826 op/s 2024-09-05T01:28:46.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:46 smithi194 ceph-mon[34941]: pgmap v757: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.8 MiB/s wr, 826 op/s 2024-09-05T01:28:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:48 smithi183 ceph-mon[34589]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.6 MiB/s wr, 953 op/s 2024-09-05T01:28:48.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:48 smithi129 ceph-mon[30979]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.6 MiB/s wr, 953 op/s 2024-09-05T01:28:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:48 smithi194 ceph-mon[34941]: pgmap v758: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.6 MiB/s wr, 953 op/s 2024-09-05T01:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 -- 172.21.15.129:0/874345337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 msgr2=0x7fcb741107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 --2- 172.21.15.129:0/874345337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741107e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009930 tx=0x7fcb6802a440 comp rx=0 tx=0).stop 2024-09-05T01:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 -- 172.21.15.129:0/874345337 shutdown_connections 2024-09-05T01:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 --2- 172.21.15.129:0/874345337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741107e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.315 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 --2- 172.21.15.129:0/874345337 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb74109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 --2- 172.21.15.129:0/874345337 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 0x7fcb74073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 -- 172.21.15.129:0/874345337 >> 172.21.15.129:0/874345337 conn(0x7fcb7406ed10 msgr2=0x7fcb74071150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 -- 172.21.15.129:0/874345337 shutdown_connections 2024-09-05T01:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.314+0000 7fcb79df1640 1 -- 172.21.15.129:0/874345337 wait complete. 2024-09-05T01:28:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 Processor -- start 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 -- start start 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 0x7fcb741a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb741a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74113120 con 0x7fcb74109590 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74112fa0 con 0x7fcb74109f50 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb79df1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb741132a0 con 0x7fcb74073560 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb78def640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 0x7fcb741a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb741a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb795f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb741a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:49654/0 (socket says 172.21.15.129:49654) 2024-09-05T01:28:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.315+0000 7fcb73fff640 1 -- 172.21.15.129:0/3210778827 learned_addr learned my addr 172.21.15.129:0/3210778827 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 -- 172.21.15.129:0/3210778827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 msgr2=0x7fcb741a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 0x7fcb741a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 -- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 msgr2=0x7fcb741a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb741a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 -- 172.21.15.129:0/3210778827 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb741ae5e0 con 0x7fcb74109f50 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.316+0000 7fcb795f0640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741adf70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009230 tx=0x7fcb68003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.317+0000 7fcb71ffb640 1 -- 172.21.15.129:0/3210778827 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6803f070 con 0x7fcb74109f50 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.317+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb741ae870 con 0x7fcb74109f50 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.317+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb741aed80 con 0x7fcb74109f50 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.317+0000 7fcb71ffb640 1 -- 172.21.15.129:0/3210778827 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb68031c30 con 0x7fcb74109f50 2024-09-05T01:28:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.317+0000 7fcb71ffb640 1 -- 172.21.15.129:0/3210778827 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb68035030 con 0x7fcb74109f50 2024-09-05T01:28:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.323+0000 7fcb7ddf1740 1 Processor -- start 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb71ffb640 1 -- 172.21.15.129:0/3210778827 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcb68042070 con 0x7fcb74109f50 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 -- start start 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7329f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d732ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f6d72882f0 con 0x55f6d732a540 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6d7230270 con 0x55f6d7324b70 2024-09-05T01:28:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.324+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f6d72842b0 con 0x55f6d73254b0 2024-09-05T01:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50488/0 (socket says 172.21.15.129:50488) 2024-09-05T01:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 -- 172.21.15.129:0/153549494 learned_addr learned my addr 172.21.15.129:0/153549494 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d732ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb78def640 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7329f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb71ffb640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 0x7fcb5c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 -- 172.21.15.129:0/153549494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 msgr2=0x55f6d7329f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7329f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 -- 172.21.15.129:0/153549494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 msgr2=0x55f6d732ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d732ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb71ffb640 1 -- 172.21.15.129:0/3210778827 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcb680efaf0 con 0x7fcb74109f50 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.325+0000 7fcb73fff640 1 -- 172.21.15.129:0/153549494 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d722f340 con 0x55f6d7324b70 2024-09-05T01:28:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.326+0000 7fcb73fff640 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c013990 tx=0x7fcb6c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15f0184db3d748b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.326+0000 7fcb795f0640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 0x7fcb5c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.326+0000 7fcb5affd640 1 -- 172.21.15.129:0/153549494 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c011a80 con 0x55f6d7324b70 2024-09-05T01:28:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.326+0000 7fcb5affd640 1 -- 172.21.15.129:0/153549494 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c011c20 con 0x55f6d7324b70 2024-09-05T01:28:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.326+0000 7fcb5affd640 1 -- 172.21.15.129:0/153549494 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c00f5c0 con 0x55f6d7324b70 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/153549494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 msgr2=0x55f6d7324f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c013990 tx=0x7fcb6c009b60 comp rx=0 tx=0).stop 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb795f0640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 0x7fcb5c0a50b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb680bbf20 tx=0x7fcb68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/153549494 shutdown_connections 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d732ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7329f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/153549494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d7324f70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.327+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/153549494 >> 172.21.15.129:0/153549494 conn(0x55f6d73201e0 msgr2=0x55f6d73205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/153549494 shutdown_connections 2024-09-05T01:28:49.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/153549494 wait complete. 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 -- start start 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 Processor -- start 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d732ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.328+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d73363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f6d732d510 con 0x55f6d732a540 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f6d7211290 con 0x55f6d7324b70 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55f6d732d690 con 0x55f6d73254b0 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb7ddf1740 10 client.0 _ll_register_callbacks cb 0x55f6d7242140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:49.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb73fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d732ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:33588/0 (socket says 172.21.15.129:33588) 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 -- 172.21.15.129:0/1855376153 learned_addr learned my addr 172.21.15.129:0/1855376153 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d73363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 -- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 msgr2=0x55f6d732ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d732ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 -- 172.21.15.129:0/1855376153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 msgr2=0x55f6d73363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d73363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb78def640 1 -- 172.21.15.129:0/1855376153 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d6f9b210 con 0x55f6d73254b0 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.329+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d732ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb78def640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb640126e0 tx=0x7fcb64012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac764940e1b02de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb53fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640120d0 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcb6400f920 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6c011c20 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb64012270 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb7ddf1740 10 client.26185 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.330+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f6d7067620 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6c011c20 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcb64026ce0 con 0x55f6d73254b0 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 10 client.26185 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 10 client.26185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 20 client.26185 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d733ab70 0x55f6d733cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 4 client.26185 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:28:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6d7295a20 con 0x55f6d733ab70 2024-09-05T01:28:49.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.331+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d733ab70 0x55f6d733cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.332+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d733ab70 0x55f6d733cf90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b74443b0eaa80c server_cookie=5f07e6d9036b2dcd in_seq=0 out_seq=0 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.332+0000 7fcb53fff640 10 client.26185 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb53fff640 1 -- 172.21.15.129:0/1855376153 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6d7295a20 con 0x55f6d733ab70 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb53fff640 10 client.26185 handle_command_reply: tid=0 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 client.26185 shutdown 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 msgr2=0x55f6d732f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcb640126e0 tx=0x7fcb64012ba0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 shutdown_connections 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d733ab70 0x55f6d733cf90 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d73363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d732f3d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb53fff640 0 client.26185 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1855376153 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d732ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 >> 172.21.15.129:0/1855376153 conn(0x55f6d73201e0 msgr2=0x55f6d73219f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 shutdown_connections 2024-09-05T01:28:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.333+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1855376153 wait complete. 2024-09-05T01:28:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.334+0000 7fcb7ddf1740 20 client.26185 trim_cache size 0 max 16384 2024-09-05T01:28:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.360+0000 7fcb7ddf1740 1 Processor -- start 2024-09-05T01:28:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.360+0000 7fcb7ddf1740 1 -- start start 2024-09-05T01:28:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d73205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d7320b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60012750 con 0x55f6d732a540 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb60010020 con 0x55f6d7324b70 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6000de70 con 0x55f6d73254b0 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb78def640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb78def640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:33602/0 (socket says 172.21.15.129:33602) 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb78def640 1 -- 172.21.15.129:0/2796177810 learned_addr learned my addr 172.21.15.129:0/2796177810 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb73fff640 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d73205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb73fff640 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d7320b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb78def640 1 -- 172.21.15.129:0/2796177810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 msgr2=0x55f6d73205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.361+0000 7fcb78def640 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d73205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.362+0000 7fcb78def640 1 -- 172.21.15.129:0/2796177810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 msgr2=0x55f6d7320b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.362+0000 7fcb78def640 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d7320b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.362+0000 7fcb78def640 1 -- 172.21.15.129:0/2796177810 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6d7320000 con 0x55f6d73254b0 2024-09-05T01:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.362+0000 7fcb78def640 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009de0 tx=0x7fcb64009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602f1c3ee913d078 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.363+0000 7fcb5a7fc640 1 -- 172.21.15.129:0/2796177810 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640107b0 con 0x55f6d73254b0 2024-09-05T01:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.363+0000 7fcb5a7fc640 1 -- 172.21.15.129:0/2796177810 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6400f920 con 0x55f6d73254b0 2024-09-05T01:28:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.363+0000 7fcb5a7fc640 1 -- 172.21.15.129:0/2796177810 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64005480 con 0x55f6d73254b0 2024-09-05T01:28:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/2796177810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 msgr2=0x55f6d7336360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009de0 tx=0x7fcb64009f00 comp rx=0 tx=0).stop 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/2796177810 shutdown_connections 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d732a540 0x55f6d7320b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d73254b0 0x55f6d7336360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/2796177810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d7324b70 0x55f6d73205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/2796177810 >> 172.21.15.129:0/2796177810 conn(0x55f6d73201e0 msgr2=0x55f6d732e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/2796177810 shutdown_connections 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/2796177810 wait complete. 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 -- start start 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.364+0000 7fcb7ddf1740 1 Processor -- start 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d7324b70 0x55f6d73339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d732a540 0x55f6d7334ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6000e4e0 con 0x55f6d7324b70 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6000f950 con 0x55f6d73254b0 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcb6000ebb0 con 0x55f6d732a540 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb7ddf1740 10 client.0 _ll_register_callbacks cb 0x7fcb64009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb78def640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb78def640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:50514/0 (socket says 172.21.15.129:50514) 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb78def640 1 -- 172.21.15.129:0/1563354914 learned_addr learned my addr 172.21.15.129:0/1563354914 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.365+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d732a540 0x55f6d7334ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d7324b70 0x55f6d73339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 -- 172.21.15.129:0/1563354914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d732a540 msgr2=0x55f6d7334ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d732a540 0x55f6d7334ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 -- 172.21.15.129:0/1563354914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d7324b70 msgr2=0x55f6d73339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:28:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d7324b70 0x55f6d73339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 -- 172.21.15.129:0/1563354914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6d7335500 con 0x55f6d73254b0 2024-09-05T01:28:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb78def640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb64028050 tx=0x7fcb6400ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42d143fa6671edc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:28:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb53fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64056060 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.366+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcb640051c0 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb6400f920 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcb640053c0 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 10 client.26324 fetch_fsmap learned FSMap version 34 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f6d73a5810 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb6400f920 con 0x55f6d73254b0 2024-09-05T01:28:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcb64026470 con 0x55f6d73254b0 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 10 client.26324 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 10 client.26324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 20 client.26324 populate_metadata read hostname 'smithi129' 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d73a5f30 0x55f6d73a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 4 client.26324 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.367+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6d728fd70 con 0x55f6d73a5f30 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.369+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d73a5f30 0x55f6d73a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.369+0000 7fcb73fff640 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d73a5f30 0x55f6d73a8350 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cf06911e35671e server_cookie=f977c1eee4df2e1d in_seq=0 out_seq=0 2024-09-05T01:28:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.369+0000 7fcb53fff640 10 client.26324 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.387+0000 7fcb53fff640 1 -- 172.21.15.129:0/1563354914 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6d728fd70 con 0x55f6d73a5f30 2024-09-05T01:28:49.388 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.387+0000 7fcb53fff640 10 client.26324 handle_command_reply: tid=0 2024-09-05T01:28:49.389 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.388+0000 7fcb7ddf1740 1 client.26324 shutdown 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.389+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 msgr2=0x55f6d7333ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.389+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb64028050 tx=0x7fcb6400ffd0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 shutdown_connections 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55f6d73a5f30 0x55f6d73a8350 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55f6d732a540 0x55f6d7334ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f6d73254b0 0x55f6d7333ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 --2- 172.21.15.129:0/1563354914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f6d7324b70 0x55f6d73339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.390 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 >> 172.21.15.129:0/1563354914 conn(0x55f6d73201e0 msgr2=0x55f6d731ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb53fff640 0 client.26324 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:28:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.390+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 shutdown_connections 2024-09-05T01:28:49.391 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.391+0000 7fcb7ddf1740 1 -- 172.21.15.129:0/1563354914 wait complete. 2024-09-05T01:28:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.393+0000 7fcb7ddf1740 20 client.26324 trim_cache size 0 max 16384 2024-09-05T01:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.396+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 msgr2=0x7fcb5c0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.396+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 0x7fcb5c0a50b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcb680bbf20 tx=0x7fcb68009260 comp rx=0 tx=0).stop 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.396+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 msgr2=0x7fcb741adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.396+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741adf70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb68009230 tx=0x7fcb68003b90 comp rx=0 tx=0).stop 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 shutdown_connections 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcb5c0a2bf0 0x7fcb5c0a50b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb74109f50 0x7fcb741adf70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcb74109590 0x7fcb741a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 --2- 172.21.15.129:0/3210778827 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcb74073560 0x7fcb741a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 >> 172.21.15.129:0/3210778827 conn(0x7fcb7406ed10 msgr2=0x7fcb74115ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 shutdown_connections 2024-09-05T01:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:28:49.397+0000 7fcb79df1640 1 -- 172.21.15.129:0/3210778827 wait complete. 2024-09-05T01:28:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 705.0 s, 873.8 tps, lat 114.223 ms stddev 77.397 2024-09-05T01:28:50.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:50 smithi183 ceph-mon[34589]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 827 op/s 2024-09-05T01:28:50.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:50 smithi129 ceph-mon[30979]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 827 op/s 2024-09-05T01:28:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:50 smithi194 ceph-mon[34941]: pgmap v759: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.0 MiB/s wr, 827 op/s 2024-09-05T01:28:52.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:52 smithi183 ceph-mon[34589]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-09-05T01:28:52.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:52 smithi129 ceph-mon[30979]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-09-05T01:28:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:52 smithi194 ceph-mon[34941]: pgmap v760: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.7 MiB/s wr, 877 op/s 2024-09-05T01:28:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:54 smithi183 ceph-mon[34589]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 886 op/s 2024-09-05T01:28:54.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:54 smithi129 ceph-mon[30979]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 886 op/s 2024-09-05T01:28:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:54 smithi194 ceph-mon[34941]: pgmap v761: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 8.4 MiB/s wr, 886 op/s 2024-09-05T01:28:55.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 710.0 s, 755.6 tps, lat 129.792 ms stddev 112.242 2024-09-05T01:28:56.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:56 smithi183 ceph-mon[34589]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 794 op/s 2024-09-05T01:28:56.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:56 smithi129 ceph-mon[30979]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 794 op/s 2024-09-05T01:28:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:56 smithi194 ceph-mon[34941]: pgmap v762: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.6 MiB/s wr, 794 op/s 2024-09-05T01:28:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:28:58 smithi183 ceph-mon[34589]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 894 op/s 2024-09-05T01:28:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:28:58 smithi129 ceph-mon[30979]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 894 op/s 2024-09-05T01:28:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:28:58 smithi194 ceph-mon[34941]: pgmap v763: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.3 MiB/s wr, 894 op/s 2024-09-05T01:29:00.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 715.0 s, 705.4 tps, lat 144.951 ms stddev 149.251 2024-09-05T01:29:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:00 smithi183 ceph-mon[34589]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 790 op/s 2024-09-05T01:29:00.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:00 smithi129 ceph-mon[30979]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 790 op/s 2024-09-05T01:29:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:00 smithi194 ceph-mon[34941]: pgmap v764: 129 pgs: 129 active+clean; 1.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 790 op/s 2024-09-05T01:29:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:02 smithi183 ceph-mon[34589]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-09-05T01:29:02.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:02 smithi129 ceph-mon[30979]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-09-05T01:29:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:02 smithi194 ceph-mon[34941]: pgmap v765: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.0 MiB/s wr, 817 op/s 2024-09-05T01:29:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.556+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/3239241515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff410adc0 msgr2=0x7f5ff4111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.556+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/3239241515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff410adc0 0x7f5ff4111650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009ca0 tx=0x7f5fec02a8d0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/3239241515 shutdown_connections 2024-09-05T01:29:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/3239241515 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff410adc0 0x7f5ff4111650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/3239241515 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff410a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/3239241515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff40733b0 0x7f5ff40737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/3239241515 >> 172.21.15.129:0/3239241515 conn(0x7f5ff406ece0 msgr2=0x7f5ff4071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.558+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/3239241515 shutdown_connections 2024-09-05T01:29:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.559+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/3239241515 wait complete. 2024-09-05T01:29:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 Processor -- start 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 -- start start 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 0x7f5ff41a6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff41a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 0x7f5ff41ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113d60 con 0x7f5ff410adc0 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113be0 con 0x7f5ff410a400 2024-09-05T01:29:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.560+0000 7f5ffa6eb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113ee0 con 0x7f5ff40733b0 2024-09-05T01:29:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff41a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff8c88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 0x7f5ff41ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff8c88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 0x7f5ff41ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40564/0 (socket says 172.21.15.129:40564) 2024-09-05T01:29:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff8c88640 1 -- 172.21.15.129:0/2834389157 learned_addr learned my addr 172.21.15.129:0/2834389157 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 0x7f5ff41a6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 -- 172.21.15.129:0/2834389157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 msgr2=0x7f5ff41a6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 0x7f5ff41a6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 -- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 msgr2=0x7f5ff41ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 0x7f5ff41ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.561+0000 7f5ff37fe640 1 -- 172.21.15.129:0/2834389157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff4119430 con 0x7f5ff410a400 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.562+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff41a71a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800b3f0 tx=0x7f5fe800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.562+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 0x7f5ff41a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.562+0000 7f5ff17fa640 1 -- 172.21.15.129:0/2834389157 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8015020 con 0x7f5ff410a400 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.563+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff4119720 con 0x7f5ff410a400 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.563+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff4119cf0 con 0x7f5ff410a400 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.563+0000 7f5ff17fa640 1 -- 172.21.15.129:0/2834389157 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fe8011ce0 con 0x7f5ff410a400 2024-09-05T01:29:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.563+0000 7f5ff17fa640 1 -- 172.21.15.129:0/2834389157 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8010d10 con 0x7f5ff410a400 2024-09-05T01:29:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.570+0000 7f5ff17fa640 1 -- 172.21.15.129:0/2834389157 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5fe8018020 con 0x7f5ff410a400 2024-09-05T01:29:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.571+0000 7f5ffd48a740 1 Processor -- start 2024-09-05T01:29:02.572 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 -- start start 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e557da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e55f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56503e4bb2f0 con 0x56503e5582e0 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56503e463270 con 0x56503e55d370 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56503e4b72b0 con 0x56503e5579a0 2024-09-05T01:29:02.573 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.572+0000 7f5ff17fa640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 0x7f5fdc0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.573+0000 7f5ff17fa640 1 -- 172.21.15.129:0/2834389157 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5fe80ceb80 con 0x7f5ff410a400 2024-09-05T01:29:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.574 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40572/0 (socket says 172.21.15.129:40572) 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2187747891 learned_addr learned my addr 172.21.15.129:0/2187747891 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e557da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e55f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2187747891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 msgr2=0x56503e557da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e557da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2187747891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 msgr2=0x56503e55f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e55f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.575 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.574+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2187747891 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503e462340 con 0x56503e5582e0 2024-09-05T01:29:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.575+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 0x7f5fdc0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.575+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007800 tx=0x7f5fe4007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0476baec0ed23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.576+0000 7f5fe27fc640 1 -- 172.21.15.129:0/2187747891 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400d950 con 0x56503e5582e0 2024-09-05T01:29:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.576+0000 7f5fe27fc640 1 -- 172.21.15.129:0/2187747891 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fe4018070 con 0x56503e5582e0 2024-09-05T01:29:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.576+0000 7f5fe27fc640 1 -- 172.21.15.129:0/2187747891 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4013e00 con 0x56503e5582e0 2024-09-05T01:29:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.576+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 0x7f5fdc0a5330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098a0 tx=0x7f5fec02a8d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.577+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2187747891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 msgr2=0x56503e55cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.577+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007800 tx=0x7f5fe4007cc0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.578+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2187747891 shutdown_connections 2024-09-05T01:29:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.578+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e55f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.578+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e55cda0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.578+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2187747891 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e557da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.578+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2187747891 >> 172.21.15.129:0/2187747891 conn(0x56503e553010 msgr2=0x56503e553410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.579+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2187747891 shutdown_connections 2024-09-05T01:29:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.579+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2187747891 wait complete. 2024-09-05T01:29:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.579+0000 7f5ffd48a740 1 -- start start 2024-09-05T01:29:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 Processor -- start 2024-09-05T01:29:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5582e0 0x56503e5621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 0x56503e569190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56503e560340 con 0x56503e55d370 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56503e444290 con 0x56503e5579a0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56503e5604c0 con 0x56503e5582e0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ffd48a740 10 client.0 _ll_register_callbacks cb 0x56503e475140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40608/0 (socket says 172.21.15.129:40608) 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2097737266 learned_addr learned my addr 172.21.15.129:0/2097737266 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5582e0 0x56503e5621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.580+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 0x56503e569190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2097737266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5582e0 msgr2=0x56503e5621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5582e0 0x56503e5621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2097737266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 msgr2=0x56503e569190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 0x56503e569190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2097737266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503e1ce210 con 0x56503e5579a0 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 0x56503e569190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40077d0 tx=0x7f5fe4013ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95c6c0a0003b7ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5fd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:29:02.583 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400db50 con 0x56503e5579a0 2024-09-05T01:29:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5fe400dcf0 con 0x56503e5579a0 2024-09-05T01:29:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.581+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe4018070 con 0x56503e5579a0 2024-09-05T01:29:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.582+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fe402f050 con 0x56503e5579a0 2024-09-05T01:29:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.584+0000 7f5ffd48a740 10 client.26333 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.584+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56503e29a620 con 0x56503e5579a0 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fe4018070 con 0x56503e5579a0 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe401c3f0 con 0x56503e5579a0 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 10 client.26333 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 10 client.26333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 20 client.26333 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e56d600 0x56503e56f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 4 client.26333 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.585+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56503e4c8a20 con 0x56503e56d600 2024-09-05T01:29:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.586+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e56d600 0x56503e56f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.587+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e56d600 0x56503e56f9f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c63d0a9a8d7361 server_cookie=3f1af9ce8ba1e6de in_seq=0 out_seq=0 2024-09-05T01:29:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.587+0000 7f5fd77fe640 10 client.26333 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.588+0000 7f5fd77fe640 1 -- 172.21.15.129:0/2097737266 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56503e4c8a20 con 0x56503e56d600 2024-09-05T01:29:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.588+0000 7f5fd77fe640 10 client.26333 handle_command_reply: tid=0 2024-09-05T01:29:02.589 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.589+0000 7f5ffd48a740 1 client.26333 shutdown 2024-09-05T01:29:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.589+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 msgr2=0x56503e561c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.590 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.589+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fe40077d0 tx=0x7f5fe4013ef0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 shutdown_connections 2024-09-05T01:29:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e56d600 0x56503e56f9f0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.591 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e55d370 0x56503e569190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5582e0 0x56503e5621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5fd77fe640 0 client.26333 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2097737266 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e5579a0 0x56503e561c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 >> 172.21.15.129:0/2097737266 conn(0x56503e553010 msgr2=0x56503e554820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 shutdown_connections 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2097737266 wait complete. 2024-09-05T01:29:02.592 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.590+0000 7f5ffd48a740 20 client.26333 trim_cache size 0 max 16384 2024-09-05T01:29:02.626 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.626+0000 7f5ffd48a740 1 Processor -- start 2024-09-05T01:29:02.627 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.626+0000 7f5ffd48a740 1 -- start start 2024-09-05T01:29:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.626+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.626+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e5690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e561f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.628 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8012770 con 0x56503e5582e0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd8010040 con 0x56503e55d370 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd800de90 con 0x56503e5579a0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:48710/0 (socket says 172.21.15.129:48710) 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2056950689 learned_addr learned my addr 172.21.15.129:0/2056950689 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e5690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e561f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2056950689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 msgr2=0x56503e561f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e561f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2056950689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 msgr2=0x56503e5690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e5690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 -- 172.21.15.129:0/2056950689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503e5587c0 con 0x56503e5579a0 2024-09-05T01:29:02.629 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e561f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007e20 tx=0x7f5fe4007300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcaa31d5e0b65cb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.627+0000 7f5fe1ffb640 1 -- 172.21.15.129:0/2056950689 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe401c3f0 con 0x56503e5579a0 2024-09-05T01:29:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5fe1ffb640 1 -- 172.21.15.129:0/2056950689 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fe4018040 con 0x56503e5579a0 2024-09-05T01:29:02.630 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5fe1ffb640 1 -- 172.21.15.129:0/2056950689 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4024420 con 0x56503e5579a0 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2056950689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 msgr2=0x56503e561a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007e20 tx=0x7f5fe4007300 comp rx=0 tx=0).stop 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2056950689 shutdown_connections 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e561f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e5690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/2056950689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e561a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.628+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2056950689 >> 172.21.15.129:0/2056950689 conn(0x56503e553010 msgr2=0x56503e5594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2056950689 shutdown_connections 2024-09-05T01:29:02.633 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 -- 172.21.15.129:0/2056950689 wait complete. 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 -- start start 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 Processor -- start 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e566770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.629+0000 7f5ffd48a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e567cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd800e500 con 0x56503e5582e0 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd800f970 con 0x56503e55d370 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ffd48a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd800ebd0 con 0x56503e5579a0 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:40598/0 (socket says 172.21.15.129:40598) 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff37fe640 1 -- 172.21.15.129:0/3842883432 learned_addr learned my addr 172.21.15.129:0/3842883432 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:02.634 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e566770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ffd48a740 10 client.0 _ll_register_callbacks cb 0x7f5fec005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff8c88640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e567cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 -- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 msgr2=0x56503e567cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e567cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 -- 172.21.15.129:0/3842883432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 msgr2=0x56503e566cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 -- 172.21.15.129:0/3842883432 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503e5682d0 con 0x56503e5579a0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5ff3fff640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e566770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007370 tx=0x7f5fe4006c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2de7d206e5a3f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.630+0000 7f5fd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe401c910 con 0x56503e5579a0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f5fe401c3f0 con 0x56503e5579a0 2024-09-05T01:29:02.635 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe4018040 con 0x56503e5579a0 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5fe4013750 con 0x56503e5579a0 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5ffd48a740 10 client.26209 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.631+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56503e5d81f0 con 0x56503e5579a0 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fe4018040 con 0x56503e5579a0 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe4006250 con 0x56503e5579a0 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 10 client.26209 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 20 client.26209 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e5d8a80 0x56503e5dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 4 client.26209 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:29:02.636 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.632+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56503e4c2d70 con 0x56503e5d8a80 2024-09-05T01:29:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.633+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e5d8a80 0x56503e5dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.633+0000 7f5fd77fe640 10 client.26209 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:02.637 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.633+0000 7f5ff37fe640 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e5d8a80 0x56503e5dae70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab2e04580792419 server_cookie=a1144a6ff1f74525 in_seq=0 out_seq=0 2024-09-05T01:29:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.688+0000 7f5fd77fe640 1 -- 172.21.15.129:0/3842883432 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56503e4c2d70 con 0x56503e5d8a80 2024-09-05T01:29:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.688+0000 7f5fd77fe640 10 client.26209 handle_command_reply: tid=0 2024-09-05T01:29:02.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.689+0000 7f5ffd48a740 1 client.26209 shutdown 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 msgr2=0x56503e566770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e566770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4007370 tx=0x7f5fe4006c10 comp rx=0 tx=0).stop 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 shutdown_connections 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56503e5d8a80 0x56503e5dae70 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56503e55d370 0x56503e567cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56503e5582e0 0x56503e566cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 --2- 172.21.15.129:0/3842883432 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56503e5579a0 0x56503e566770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5fd77fe640 0 client.26209 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.691+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 >> 172.21.15.129:0/3842883432 conn(0x56503e553010 msgr2=0x56503e5519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.692+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 shutdown_connections 2024-09-05T01:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.692+0000 7f5ffd48a740 1 -- 172.21.15.129:0/3842883432 wait complete. 2024-09-05T01:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.693+0000 7f5ffd48a740 20 client.26209 trim_cache size 0 max 16384 2024-09-05T01:29:02.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:29:02.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:29:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.695+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 msgr2=0x7f5fdc0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.695 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.695+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 0x7f5fdc0a5330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0098a0 tx=0x7f5fec02a8d0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.695+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 msgr2=0x7f5ff41a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:02.696 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.695+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff41a71a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800b3f0 tx=0x7f5fe800b8b0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 shutdown_connections 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f5fdc0a2e70 0x7f5fdc0a5330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ff410adc0 0x7f5ff41ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ff410a400 0x7f5ff41a71a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 --2- 172.21.15.129:0/2834389157 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5ff40733b0 0x7f5ff41a6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:02.697 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.696+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 >> 172.21.15.129:0/2834389157 conn(0x7f5ff406ece0 msgr2=0x7f5ff41168b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.698+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 shutdown_connections 2024-09-05T01:29:02.699 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:02.698+0000 7f5ffa6eb640 1 -- 172.21.15.129:0/2834389157 wait complete. 2024-09-05T01:29:04.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:04 smithi183 ceph-mon[34589]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 836 op/s 2024-09-05T01:29:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:04 smithi129 ceph-mon[30979]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 836 op/s 2024-09-05T01:29:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:04 smithi194 ceph-mon[34941]: pgmap v766: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 836 op/s 2024-09-05T01:29:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 720.0 s, 887.2 tps, lat 112.972 ms stddev 71.500 2024-09-05T01:29:06.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:06 smithi183 ceph-mon[34589]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-05T01:29:06.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:06 smithi129 ceph-mon[30979]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-05T01:29:06.503 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:06 smithi194 ceph-mon[34941]: pgmap v767: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.3 MiB/s wr, 736 op/s 2024-09-05T01:29:06.852 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-05T01:29:06.852 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:29:07.327 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:08 smithi183 ceph-mon[34589]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 8.3 MiB/s wr, 860 op/s 2024-09-05T01:29:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:08 smithi129 ceph-mon[30979]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 8.3 MiB/s wr, 860 op/s 2024-09-05T01:29:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:08 smithi194 ceph-mon[34941]: pgmap v768: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 8.3 MiB/s wr, 860 op/s 2024-09-05T01:29:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 725.0 s, 840.2 tps, lat 118.991 ms stddev 70.304 2024-09-05T01:29:10.410 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:10 smithi129 ceph-mon[30979]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 7.7 MiB/s wr, 789 op/s 2024-09-05T01:29:10.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:10 smithi183 ceph-mon[34589]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 7.7 MiB/s wr, 789 op/s 2024-09-05T01:29:10.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:10 smithi194 ceph-mon[34941]: pgmap v769: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 7.7 MiB/s wr, 789 op/s 2024-09-05T01:29:11.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.926+0000 7fa40affd640 1 -- 172.21.15.129:0/2807678426 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404030d20 con 0x7fa40c077040 2024-09-05T01:29:11.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.927+0000 7fa41296a640 1 -- 172.21.15.129:0/2807678426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 msgr2=0x7fa40c1124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:11.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.927+0000 7fa41296a640 1 --2- 172.21.15.129:0/2807678426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 0x7fa40c1124e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa404009be0 tx=0x7fa40402a6d0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.930+0000 7fa41296a640 1 -- 172.21.15.129:0/2807678426 shutdown_connections 2024-09-05T01:29:11.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.930+0000 7fa41296a640 1 --2- 172.21.15.129:0/2807678426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c112a20 0x7fa40c114e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.930+0000 7fa41296a640 1 --2- 172.21.15.129:0/2807678426 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 0x7fa40c1124e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.930+0000 7fa41296a640 1 --2- 172.21.15.129:0/2807678426 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa40c076720 0x7fa40c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.931 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.930+0000 7fa41296a640 1 -- 172.21.15.129:0/2807678426 >> 172.21.15.129:0/2807678426 conn(0x7fa40c0ffa60 msgr2=0x7fa40c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:11.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.932+0000 7fa41296a640 1 -- 172.21.15.129:0/2807678426 shutdown_connections 2024-09-05T01:29:11.933 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.932+0000 7fa41296a640 1 -- 172.21.15.129:0/2807678426 wait complete. 2024-09-05T01:29:11.934 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.933+0000 7fa41296a640 1 Processor -- start 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.933+0000 7fa41296a640 1 -- start start 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 0x7fa40c108d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa40c112a20 0x7fa40c0730f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c104590 con 0x7fa40c112a20 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c104410 con 0x7fa40c076720 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa41296a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c104710 con 0x7fa40c077040 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:11.935 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa40bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40642/0 (socket says 172.21.15.129:40642) 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.934+0000 7fa40bfff640 1 -- 172.21.15.129:0/1065652507 learned_addr learned my addr 172.21.15.129:0/1065652507 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 -- 172.21.15.129:0/1065652507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 msgr2=0x7fa40c108d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 0x7fa40c108d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 -- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa40c112a20 msgr2=0x7fa40c0730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa40c112a20 0x7fa40c0730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:11.936 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 -- 172.21.15.129:0/1065652507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c073870 con 0x7fa40c076720 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa40bfff640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc00ccc0 con 0x7fa40c076720 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3fc0054d0 con 0x7fa40c076720 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.935+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc010430 con 0x7fa40c076720 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.936+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c073b60 con 0x7fa40c076720 2024-09-05T01:29:11.937 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.936+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c110700 con 0x7fa40c076720 2024-09-05T01:29:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.942+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa3fc0046c0 con 0x7fa40c076720 2024-09-05T01:29:11.943 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.940+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40c076b00 con 0x7fa40c076720 2024-09-05T01:29:11.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.944+0000 7fa4097fa640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 0x7fa3f40a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:11.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.944+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa3fc0cf7c0 con 0x7fa40c076720 2024-09-05T01:29:11.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.944+0000 7fa40b7fe640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 0x7fa3f40a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:11.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.945+0000 7fa40b7fe640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 0x7fa3f40a51d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa4040099a0 tx=0x7fa404005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:11.950 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:11.949+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3fc09c310 con 0x7fa40c076720 2024-09-05T01:29:12.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:12 smithi129 ceph-mon[30979]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-05T01:29:12.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.336+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa40c06d350 con 0x7fa40c076720 2024-09-05T01:29:12.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.337+0000 7fa4097fa640 1 -- 172.21.15.129:0/1065652507 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7fa3fc09bc10 con 0x7fa40c076720 2024-09-05T01:29:12.339 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:29:12.339 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:29:12.340 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:29:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.341+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 msgr2=0x7fa3f40a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 0x7fa3f40a51d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa4040099a0 tx=0x7fa404005fd0 comp rx=0 tx=0).stop 2024-09-05T01:29:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 msgr2=0x7fa40c1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:12.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00e9c0 tx=0x7fa3fc00ee80 comp rx=0 tx=0).stop 2024-09-05T01:29:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 shutdown_connections 2024-09-05T01:29:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3f40a2d10 0x7fa3f40a51d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:12.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa40c112a20 0x7fa40c0730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:12.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa40c077040 0x7fa40c108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:12.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 --2- 172.21.15.129:0/1065652507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa40c076720 0x7fa40c1087d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:12.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 >> 172.21.15.129:0/1065652507 conn(0x7fa40c0ffa60 msgr2=0x7fa40c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:12.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.342+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 shutdown_connections 2024-09-05T01:29:12.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:12.343+0000 7fa41296a640 1 -- 172.21.15.129:0/1065652507 wait complete. 2024-09-05T01:29:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:12 smithi183 ceph-mon[34589]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-05T01:29:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:12 smithi194 ceph-mon[34941]: pgmap v770: 129 pgs: 129 active+clean; 1.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 7.9 MiB/s wr, 818 op/s 2024-09-05T01:29:13.168 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-05T01:29:13.168 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-05T01:29:13.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:13 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1065652507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:13.494 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:13 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1065652507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:13 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1065652507' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:13.633 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:14.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:14 smithi129 ceph-mon[30979]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 8.3 MiB/s wr, 856 op/s 2024-09-05T01:29:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:14 smithi194 ceph-mon[34941]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 8.3 MiB/s wr, 856 op/s 2024-09-05T01:29:14.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:14 smithi183 ceph-mon[34589]: pgmap v771: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 8.3 MiB/s wr, 856 op/s 2024-09-05T01:29:15.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 730.0 s, 849.8 tps, lat 117.922 ms stddev 47.035 2024-09-05T01:29:15.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.445+0000 7f66fe7fc640 1 -- 172.21.15.129:0/1871115823 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4040770 con 0x7f6700104990 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.446+0000 7f6705f17640 1 -- 172.21.15.129:0/1871115823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700104990 msgr2=0x7f67001089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.446+0000 7f6705f17640 1 --2- 172.21.15.129:0/1871115823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700104990 0x7f67001089e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66f401a910 tx=0x7f66f403b430 comp rx=0 tx=0).stop 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 -- 172.21.15.129:0/1871115823 shutdown_connections 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 --2- 172.21.15.129:0/1871115823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700109110 0x7f6700110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 --2- 172.21.15.129:0/1871115823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700104990 0x7f67001089e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 --2- 172.21.15.129:0/1871115823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f67001043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.449 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 -- 172.21.15.129:0/1871115823 >> 172.21.15.129:0/1871115823 conn(0x7f67000ffa20 msgr2=0x7f6700101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 -- 172.21.15.129:0/1871115823 shutdown_connections 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.447+0000 7f6705f17640 1 -- 172.21.15.129:0/1871115823 wait complete. 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 Processor -- start 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 -- start start 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.450 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104990 0x7f6700073c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700109110 0x7f6700071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f670006e4c0 con 0x7f6700103fe0 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f670006e340 con 0x7f6700109110 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.448+0000 7f6705f17640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f670006e640 con 0x7f6700104990 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53486/0 (socket says 172.21.15.129:53486) 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 -- 172.21.15.129:0/1554059305 learned_addr learned my addr 172.21.15.129:0/1554059305 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66feffd640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104990 0x7f6700073c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 -- 172.21.15.129:0/1554059305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104990 msgr2=0x7f6700073c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104990 0x7f6700073c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 -- 172.21.15.129:0/1554059305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700109110 msgr2=0x7f6700071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700109110 0x7f6700071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.449+0000 7f66ff7fe640 1 -- 172.21.15.129:0/1554059305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6700072620 con 0x7f6700103fe0 2024-09-05T01:29:15.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.450+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b4a0 tx=0x7f66f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.450+0000 7f66fcff9640 1 -- 172.21.15.129:0/1554059305 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0014070 con 0x7f6700103fe0 2024-09-05T01:29:15.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.450+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6700072910 con 0x7f6700103fe0 2024-09-05T01:29:15.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.451+0000 7f66fcff9640 1 -- 172.21.15.129:0/1554059305 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f0010ac0 con 0x7f6700103fe0 2024-09-05T01:29:15.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.451+0000 7f66fcff9640 1 -- 172.21.15.129:0/1554059305 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017c30 con 0x7f6700103fe0 2024-09-05T01:29:15.453 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.451+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670019a6e0 con 0x7f6700103fe0 2024-09-05T01:29:15.458 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.456+0000 7f66fcff9640 1 -- 172.21.15.129:0/1554059305 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f6700103fe0 2024-09-05T01:29:15.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.457+0000 7f66fcff9640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 0x7f66e80a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.458+0000 7f66fcff9640 1 -- 172.21.15.129:0/1554059305 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f66f00ceb10 con 0x7f6700103fe0 2024-09-05T01:29:15.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.458+0000 7f66feffd640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 0x7f66e80a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.459+0000 7f66feffd640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 0x7f66e80a5210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f67000737f0 tx=0x7f66f4005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 Processor -- start 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 -- start start 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82c080 0x556ffa836400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53494/0 (socket says 172.21.15.129:53494) 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa836940 0x556ffa838d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ffa7912a0 con 0x556ffa82b760 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f66ff7fe640 1 -- 172.21.15.129:0/2017556689 learned_addr learned my addr 172.21.15.129:0/2017556689 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.473+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ffa738b10 con 0x556ffa82c080 2024-09-05T01:29:15.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ffa78d260 con 0x556ffa836940 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 -- 172.21.15.129:0/2017556689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa836940 msgr2=0x556ffa838d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa836940 0x556ffa838d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 -- 172.21.15.129:0/2017556689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82c080 msgr2=0x556ffa836400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82c080 0x556ffa836400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 -- 172.21.15.129:0/2017556689 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffa502b90 con 0x556ffa82b760 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.474+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097360 tx=0x7f66f009a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3de29094d5789a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.475+0000 7f66e5ffb640 1 -- 172.21.15.129:0/2017556689 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f001e070 con 0x556ffa82b760 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.475+0000 7f66e5ffb640 1 -- 172.21.15.129:0/2017556689 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f009f2b0 con 0x556ffa82b760 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.475+0000 7f66e5ffb640 1 -- 172.21.15.129:0/2017556689 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0010c70 con 0x556ffa82b760 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 msgr2=0x556ffa82bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097360 tx=0x7f66f009a920 comp rx=0 tx=0).stop 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 shutdown_connections 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa836940 0x556ffa838d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.478 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82c080 0x556ffa836400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 --2- 172.21.15.129:0/2017556689 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa82b760 0x556ffa82bb40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.476+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 >> 172.21.15.129:0/2017556689 conn(0x556ffa827cf0 msgr2=0x556ffa8280d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.477+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 shutdown_connections 2024-09-05T01:29:15.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.477+0000 7f67082f0740 1 -- 172.21.15.129:0/2017556689 wait complete. 2024-09-05T01:29:15.479 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.477+0000 7f67082f0740 1 -- start start 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 Processor -- start 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa8339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa836940 0x556ffa833ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ffa839510 con 0x556ffa836940 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ffa7196e0 con 0x556ffa834420 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ffa839690 con 0x556ffa82b760 2024-09-05T01:29:15.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.478+0000 7f67082f0740 10 client.0 _ll_register_callbacks cb 0x556ffa73da80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.479+0000 7f66fffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.479+0000 7f66fffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:51328/0 (socket says 172.21.15.129:51328) 2024-09-05T01:29:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.479+0000 7f66fffff640 1 -- 172.21.15.129:0/2281125292 learned_addr learned my addr 172.21.15.129:0/2281125292 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.480+0000 7f66feffd640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa836940 0x556ffa833ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.480+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa8339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.480+0000 7f66fffff640 1 -- 172.21.15.129:0/2281125292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 msgr2=0x556ffa8339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66fffff640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa8339a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66fffff640 1 -- 172.21.15.129:0/2281125292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa836940 msgr2=0x556ffa833ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66fffff640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa836940 0x556ffa833ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66fffff640 1 -- 172.21.15.129:0/2281125292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffa842f70 con 0x556ffa834420 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66fffff640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e720 tx=0x7f66f800ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e0f93daa92c3bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.481+0000 7f66deffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.482+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa8339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.482+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8003b70 con 0x556ffa834420 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.482+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f66f8003d10 con 0x556ffa834420 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.482+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f009f2b0 con 0x556ffa834420 2024-09-05T01:29:15.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.482+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f66f800c830 con 0x556ffa834420 2024-09-05T01:29:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 10 client.26348 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:15.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ffa845930 con 0x556ffa834420 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f009f2b0 con 0x556ffa834420 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f66f8018430 con 0x556ffa834420 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 10 client.26348 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 10 client.26348 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 20 client.26348 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.483+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa846df0 0x556ffa849210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.484+0000 7f67082f0740 4 client.26348 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.484+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ffa79e9d0 con 0x556ffa846df0 2024-09-05T01:29:15.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.484+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa846df0 0x556ffa849210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.484+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa846df0 0x556ffa849210 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca015b3016114b92 server_cookie=2e7f9244b8ba084b in_seq=0 out_seq=0 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.484+0000 7f66deffd640 10 client.26348 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.485+0000 7f66deffd640 1 -- 172.21.15.129:0/2281125292 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ffa79e9d0 con 0x556ffa846df0 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.485+0000 7f66deffd640 10 client.26348 handle_command_reply: tid=0 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.485+0000 7f67082f0740 1 client.26348 shutdown 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 msgr2=0x556ffa842980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.488 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f66f800e720 tx=0x7f66f800ebe0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 shutdown_connections 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa846df0 0x556ffa849210 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa842980 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa836940 0x556ffa833ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 --2- 172.21.15.129:0/2281125292 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa8339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.489 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 >> 172.21.15.129:0/2281125292 conn(0x556ffa827cf0 msgr2=0x556ffa82da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f66deffd640 0 client.26348 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.486+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 shutdown_connections 2024-09-05T01:29:15.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.487+0000 7f67082f0740 1 -- 172.21.15.129:0/2281125292 wait complete. 2024-09-05T01:29:15.490 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.487+0000 7f67082f0740 20 client.26348 trim_cache size 0 max 16384 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.521+0000 7f67082f0740 1 Processor -- start 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.521+0000 7f67082f0740 1 -- start start 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa835910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa84eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b7f00 con 0x556ffa827cf0 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b57d0 con 0x556ffa834420 2024-09-05T01:29:15.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f67082f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b3620 con 0x556ffa82b760 2024-09-05T01:29:15.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53512/0 (socket says 172.21.15.129:53512) 2024-09-05T01:29:15.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66ff7fe640 1 -- 172.21.15.129:0/4165392433 learned_addr learned my addr 172.21.15.129:0/4165392433 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa84eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 -- 172.21.15.129:0/4165392433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 msgr2=0x556ffa835910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa835910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 -- 172.21.15.129:0/4165392433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 msgr2=0x556ffa842770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.522+0000 7f66fffff640 1 -- 172.21.15.129:0/4165392433 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffa823260 con 0x556ffa834420 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.523+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.523+0000 7f66fffff640 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa84eb30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000f80 tx=0x7f66f800e400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe9de0c063d151a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.523+0000 7f66e57fa640 1 -- 172.21.15.129:0/4165392433 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f802a070 con 0x556ffa834420 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.523+0000 7f66e57fa640 1 -- 172.21.15.129:0/4165392433 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f8018630 con 0x556ffa834420 2024-09-05T01:29:15.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.523+0000 7f66e57fa640 1 -- 172.21.15.129:0/4165392433 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8003bd0 con 0x556ffa834420 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 -- 172.21.15.129:0/4165392433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 msgr2=0x556ffa84eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa84eb30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66f8000f80 tx=0x7f66f800e400 comp rx=0 tx=0).stop 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 -- 172.21.15.129:0/4165392433 shutdown_connections 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa834420 0x556ffa84eb30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa82b760 0x556ffa835910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 --2- 172.21.15.129:0/4165392433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa842770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.524+0000 7f67082f0740 1 -- 172.21.15.129:0/4165392433 >> 172.21.15.129:0/4165392433 conn(0x556ffa834040 msgr2=0x556ffa835200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.525+0000 7f67082f0740 1 -- 172.21.15.129:0/4165392433 shutdown_connections 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.525+0000 7f67082f0740 1 -- 172.21.15.129:0/4165392433 wait complete. 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.525+0000 7f67082f0740 1 -- start start 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 Processor -- start 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82b760 0x556ffa83ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa834420 0x556ffa83b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b3c90 con 0x556ffa827cf0 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.526+0000 7f67082f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b5100 con 0x556ffa82b760 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.527+0000 7f67082f0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f67001b4360 con 0x556ffa834420 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.527+0000 7f67082f0740 10 client.0 _ll_register_callbacks cb 0x7f66f800e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.527+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.529 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.527+0000 7f66ff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53522/0 (socket says 172.21.15.129:53522) 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.527+0000 7f66ff7fe640 1 -- 172.21.15.129:0/678676130 learned_addr learned my addr 172.21.15.129:0/678676130 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 -- 172.21.15.129:0/678676130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa834420 msgr2=0x556ffa83b8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa834420 0x556ffa83b8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 -- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82b760 msgr2=0x556ffa83ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82b760 0x556ffa83ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 -- 172.21.15.129:0/678676130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffa831840 con 0x556ffa827cf0 2024-09-05T01:29:15.530 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.528+0000 7f66ff7fe640 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097310 tx=0x7f66f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143380b131be5e2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.529+0000 7f66deffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.529+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f001e040 con 0x556ffa827cf0 2024-09-05T01:29:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.529+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f8018630 con 0x556ffa827cf0 2024-09-05T01:29:15.531 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.529+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f66f0010c70 con 0x556ffa827cf0 2024-09-05T01:29:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.530+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f66f009f2b0 con 0x556ffa827cf0 2024-09-05T01:29:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.530+0000 7f67082f0740 10 client.16779 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:15.532 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.530+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ffa8b1fd0 con 0x556ffa827cf0 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.530+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66f8018630 con 0x556ffa827cf0 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.530+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f66f00b9b40 con 0x556ffa827cf0 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 10 client.16779 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 10 client.16779 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 20 client.16779 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa836940 0x556ffa8b4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 4 client.16779 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x556ffa798d20 con 0x556ffa836940 2024-09-05T01:29:15.533 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.531+0000 7f66feffd640 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa836940 0x556ffa8b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.532+0000 7f66deffd640 10 client.16779 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.534 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.532+0000 7f66feffd640 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa836940 0x556ffa8b4930 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed6111d8ca34b7b5 server_cookie=633a95c8299651d7 in_seq=0 out_seq=0 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 -- 172.21.15.129:0/902845974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 msgr2=0x7fd828104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/902845974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828104a40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd818009900 tx=0x7fd81802a410 comp rx=0 tx=0).stop 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 -- 172.21.15.129:0/902845974 shutdown_connections 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/902845974 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 0x7fd828109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/902845974 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 0x7fd828107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/902845974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828104a40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 -- 172.21.15.129:0/902845974 >> 172.21.15.129:0/902845974 conn(0x7fd8280fc1b0 msgr2=0x7fd8280fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 -- 172.21.15.129:0/902845974 shutdown_connections 2024-09-05T01:29:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 -- 172.21.15.129:0/902845974 wait complete. 2024-09-05T01:29:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.797+0000 7fd82dbe8640 1 Processor -- start 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 -- start start 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 0x7fd8281982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 0x7fd82819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810d6e0 con 0x7fd828102650 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810d560 con 0x7fd8281078b0 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd82dbe8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd82810d860 con 0x7fd828104f80 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53542/0 (socket says 172.21.15.129:53542) 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 -- 172.21.15.129:0/2534193354 learned_addr learned my addr 172.21.15.129:0/2534193354 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd827fff640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 0x7fd82819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 -- 172.21.15.129:0/2534193354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 msgr2=0x7fd8281982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd826ffd640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 0x7fd8281982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 0x7fd8281982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 -- 172.21.15.129:0/2534193354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 msgr2=0x7fd82819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 0x7fd82819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 -- 172.21.15.129:0/2534193354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82819f990 con 0x7fd828102650 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.798+0000 7fd8277fe640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd818005940 tx=0x7fd8180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.799+0000 7fd824ff9640 1 -- 172.21.15.129:0/2534193354 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81803f030 con 0x7fd828102650 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.799+0000 7fd824ff9640 1 -- 172.21.15.129:0/2534193354 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8180318f0 con 0x7fd828102650 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.799+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82819fc20 con 0x7fd828102650 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.799+0000 7fd824ff9640 1 -- 172.21.15.129:0/2534193354 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818035070 con 0x7fd828102650 2024-09-05T01:29:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.799+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8281a0110 con 0x7fd828102650 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd824ff9640 1 -- 172.21.15.129:0/2534193354 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd818042030 con 0x7fd828102650 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 Processor -- start 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 -- start start 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b6dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b70830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562d51acc2f0 con 0x562d51b689a0 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d51a74270 con 0x562d51b692e0 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd830a5f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562d51ac82b0 con 0x562d51b6e370 2024-09-05T01:29:15.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd827fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.801+0000 7fd827fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53556/0 (socket says 172.21.15.129:53556) 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 -- 172.21.15.129:0/1076220894 learned_addr learned my addr 172.21.15.129:0/1076220894 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b6dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd824ff9640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 0x7fd8100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b70830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 0x7fd8100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 -- 172.21.15.129:0/1076220894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 msgr2=0x562d51b70830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b70830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 -- 172.21.15.129:0/1076220894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 msgr2=0x562d51b6dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b6dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 -- 172.21.15.129:0/1076220894 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d51a73340 con 0x562d51b689a0 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd824ff9640 1 -- 172.21.15.129:0/2534193354 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd8180efd20 con 0x7fd828102650 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b70830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd827fff640 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd820013ac0 tx=0x7fd820013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f69614cf940c018d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd80dffb640 1 -- 172.21.15.129:0/1076220894 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820022020 con 0x562d51b689a0 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd80dffb640 1 -- 172.21.15.129:0/1076220894 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8200134e0 con 0x562d51b689a0 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.802+0000 7fd80dffb640 1 -- 172.21.15.129:0/1076220894 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820010b70 con 0x562d51b689a0 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- 172.21.15.129:0/1076220894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 msgr2=0x562d51b68da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd827fff640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 0x7fd8100a50f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd820023000 tx=0x7fd820021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd820013ac0 tx=0x7fd820013f80 comp rx=0 tx=0).stop 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- 172.21.15.129:0/1076220894 shutdown_connections 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b70830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b6dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1076220894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b68da0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- 172.21.15.129:0/1076220894 >> 172.21.15.129:0/1076220894 conn(0x562d51b64010 msgr2=0x562d51b64410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- 172.21.15.129:0/1076220894 shutdown_connections 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- 172.21.15.129:0/1076220894 wait complete. 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.803+0000 7fd830a5f740 1 -- start start 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 Processor -- start 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b73100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b7a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562d51b71340 con 0x562d51b689a0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562d51a55290 con 0x562d51b692e0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562d51b714c0 con 0x562d51b6e370 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd830a5f740 10 client.0 _ll_register_callbacks cb 0x562d51a86140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53558/0 (socket says 172.21.15.129:53558) 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 -- 172.21.15.129:0/1734830678 learned_addr learned my addr 172.21.15.129:0/1734830678 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b73100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd827fff640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b7a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 -- 172.21.15.129:0/1734830678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 msgr2=0x562d51b7a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b7a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 -- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 msgr2=0x562d51b73100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b73100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 -- 172.21.15.129:0/1734830678 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d517df210 con 0x562d51b689a0 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd818037fd0 tx=0x7fd818009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222fdf7861353e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.804+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd818049070 con 0x562d51b689a0 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd8180b87e0 con 0x562d51b689a0 2024-09-05T01:29:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d51ad9a20 con 0x562d51b689a0 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd81803b460 con 0x562d51b689a0 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 10 client.16791 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d518ab620 con 0x562d51b689a0 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562d51ad9a20 con 0x562d51b689a0 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd8180b6ce0 con 0x562d51b689a0 2024-09-05T01:29:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 10 client.16791 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 20 client.16791 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51b7e800 0x562d51b80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 4 client.16791 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d51ad3d70 con 0x562d51b7e800 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.805+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51b7e800 0x562d51b80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.806+0000 7fd806ffd640 10 client.16791 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.806+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51b7e800 0x562d51b80c40 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f30d541f13e9f968 server_cookie=11bbbbbfb15797b2 in_seq=0 out_seq=0 2024-09-05T01:29:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd806ffd640 1 -- 172.21.15.129:0/1734830678 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d51ad3d70 con 0x562d51b7e800 2024-09-05T01:29:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd806ffd640 10 client.16791 handle_command_reply: tid=0 2024-09-05T01:29:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 client.16791 shutdown 2024-09-05T01:29:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 msgr2=0x562d51b72bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.842 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd818037fd0 tx=0x7fd818009560 comp rx=0 tx=0).stop 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 shutdown_connections 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51b7e800 0x562d51b80c40 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b6e370 0x562d51b7a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b73100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1734830678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51b72bc0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 >> 172.21.15.129:0/1734830678 conn(0x562d51b64010 msgr2=0x562d51b65820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd806ffd640 0 client.16791 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 shutdown_connections 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 1 -- 172.21.15.129:0/1734830678 wait complete. 2024-09-05T01:29:15.843 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.840+0000 7fd830a5f740 20 client.16791 trim_cache size 0 max 16384 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 Processor -- start 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 -- start start 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b689a0 0x562d51b629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b62ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281ad070 con 0x562d51b6e370 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281aa940 con 0x562d51b692e0 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.854+0000 7fd830a5f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281a8790 con 0x562d51b689a0 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53570/0 (socket says 172.21.15.129:53570) 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 -- 172.21.15.129:0/3660239712 learned_addr learned my addr 172.21.15.129:0/3660239712 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd8277fe640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b689a0 0x562d51b629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd826ffd640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b62ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 -- 172.21.15.129:0/3660239712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b689a0 msgr2=0x562d51b629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b689a0 0x562d51b629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 -- 172.21.15.129:0/3660239712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 msgr2=0x562d51b62ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b62ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 -- 172.21.15.129:0/3660239712 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d51b5f2d0 con 0x562d51b6e370 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd826ffd640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b62ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd827fff640 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd820013a70 tx=0x7fd820009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2f480961a31cc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd80d7fa640 1 -- 172.21.15.129:0/3660239712 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820030070 con 0x562d51b6e370 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd80d7fa640 1 -- 172.21.15.129:0/3660239712 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd82000fe80 con 0x562d51b6e370 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.855+0000 7fd80d7fa640 1 -- 172.21.15.129:0/3660239712 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820022040 con 0x562d51b6e370 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- 172.21.15.129:0/3660239712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 msgr2=0x562d51b63420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd820013a70 tx=0x7fd820009500 comp rx=0 tx=0).stop 2024-09-05T01:29:15.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- 172.21.15.129:0/3660239712 shutdown_connections 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b6e370 0x562d51b63420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b692e0 0x562d51b62ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 --2- 172.21.15.129:0/3660239712 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b689a0 0x562d51b629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- 172.21.15.129:0/3660239712 >> 172.21.15.129:0/3660239712 conn(0x562d51b64010 msgr2=0x562d51b72b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- 172.21.15.129:0/3660239712 shutdown_connections 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- 172.21.15.129:0/3660239712 wait complete. 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 -- start start 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 Processor -- start 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.856+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b692e0 0x562d51be4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b6e370 0x562d51bebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281a8e00 con 0x562d51b689a0 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281aa270 con 0x562d51b6e370 2024-09-05T01:29:15.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd8281a94d0 con 0x562d51b692e0 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 10 client.0 _ll_register_callbacks cb 0x7fd8180a3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53584/0 (socket says 172.21.15.129:53584) 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 -- 172.21.15.129:0/1829598207 learned_addr learned my addr 172.21.15.129:0/1829598207 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd827fff640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b6e370 0x562d51bebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b692e0 0x562d51be4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 -- 172.21.15.129:0/1829598207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b692e0 msgr2=0x562d51be4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b692e0 0x562d51be4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 -- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b6e370 msgr2=0x562d51bebe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b6e370 0x562d51bebe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 -- 172.21.15.129:0/1829598207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d51bec490 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd8277fe640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8180c4fa0 tx=0x7fd8180b8a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dcdba856411d79 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd806ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8180d4060 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd8180c4b50 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d51a50900 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd818049040 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 10 client.16797 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.857+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562d51bed810 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562d51a50900 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd8180bc3d0 con 0x562d51b689a0 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 10 client.16797 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 20 client.16797 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51bee050 0x562d51bf0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:15.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 4 client.16797 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:29:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d51a9d820 con 0x562d51bee050 2024-09-05T01:29:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51bee050 0x562d51bf0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd826ffd640 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51bee050 0x562d51bf0470 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2823bff96592d718 server_cookie=eded64e1c14253df in_seq=0 out_seq=0 2024-09-05T01:29:15.866 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.858+0000 7fd806ffd640 10 client.16797 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd806ffd640 1 -- 172.21.15.129:0/1829598207 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d51a9d820 con 0x562d51bee050 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd806ffd640 10 client.16797 handle_command_reply: tid=0 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 client.16797 shutdown 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 msgr2=0x562d51be4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8180c4fa0 tx=0x7fd8180b8a10 comp rx=0 tx=0).stop 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 shutdown_connections 2024-09-05T01:29:15.897 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562d51bee050 0x562d51bf0470 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562d51b6e370 0x562d51bebe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd806ffd640 0 client.16797 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562d51b692e0 0x562d51be4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 --2- 172.21.15.129:0/1829598207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562d51b689a0 0x562d51be4900 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 >> 172.21.15.129:0/1829598207 conn(0x562d51b64010 msgr2=0x562d51b64ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 shutdown_connections 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 1 -- 172.21.15.129:0/1829598207 wait complete. 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.895+0000 7fd830a5f740 20 client.16797 trim_cache size 0 max 16384 2024-09-05T01:29:15.898 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:29:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:29:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:29:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:29:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 msgr2=0x7fd8100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.899 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 0x7fd8100a50f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd820023000 tx=0x7fd820021000 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 msgr2=0x7fd828197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd818005940 tx=0x7fd8180042f0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 shutdown_connections 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd8100a2c30 0x7fd8100a50f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd8281078b0 0x7fd82819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd828104f80 0x7fd8281982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 --2- 172.21.15.129:0/2534193354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd828102650 0x7fd828197d70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 >> 172.21.15.129:0/2534193354 conn(0x7fd8280fc1b0 msgr2=0x7fd828102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 shutdown_connections 2024-09-05T01:29:15.900 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:15.896+0000 7fd82dbe8640 1 -- 172.21.15.129:0/2534193354 wait complete. 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f66deffd640 1 -- 172.21.15.129:0/678676130 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556ffa798d20 con 0x556ffa836940 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f66deffd640 10 client.16779 handle_command_reply: tid=0 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 client.16779 shutdown 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 msgr2=0x556ffa83e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097310 tx=0x7f66f00046c0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 shutdown_connections 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ffa836940 0x556ffa8b4930 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ffa834420 0x556ffa83b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ffa82b760 0x556ffa83ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f66deffd640 0 client.16779 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:15.901 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 --2- 172.21.15.129:0/678676130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ffa827cf0 0x556ffa83e8e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 >> 172.21.15.129:0/678676130 conn(0x556ffa834040 msgr2=0x556ffa835a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 shutdown_connections 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 1 -- 172.21.15.129:0/678676130 wait complete. 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.899+0000 7f67082f0740 20 client.16779 trim_cache size 0 max 16384 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:29:15.902 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 50, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "e400dc6a": { 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 50, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:29:15.903 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:15.904 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 msgr2=0x7f66e80a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 0x7f66e80a5210 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f67000737f0 tx=0x7f66f4005d50 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 msgr2=0x7f6700110c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f66f000b4a0 tx=0x7f66f000b960 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 shutdown_connections 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f66e80a2d50 0x7f66e80a5210 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6700109110 0x7f6700071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6700104990 0x7f6700073c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 --2- 172.21.15.129:0/1554059305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6700103fe0 0x7f6700110c30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 >> 172.21.15.129:0/1554059305 conn(0x7f67000ffa20 msgr2=0x7f6700100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 shutdown_connections 2024-09-05T01:29:15.905 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:15.900+0000 7f6705f17640 1 -- 172.21.15.129:0/1554059305 wait complete. 2024-09-05T01:29:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:16 smithi183 ceph-mon[34589]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 6.8 MiB/s wr, 730 op/s 2024-09-05T01:29:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:16 smithi129 ceph-mon[30979]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 6.8 MiB/s wr, 730 op/s 2024-09-05T01:29:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:16 smithi194 ceph-mon[34941]: pgmap v772: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 6.8 MiB/s wr, 730 op/s 2024-09-05T01:29:16.686 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e400dc6a', quiesce took 9.8 seconds. Will release after: 0.1999999999999993 2024-09-05T01:29:16.887 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:29:17.139 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 -- 172.21.15.129:0/2228996554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 msgr2=0x7f8be010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:17.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 --2- 172.21.15.129:0/2228996554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be010e5f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009ca0 tx=0x7f8bd802a950 comp rx=0 tx=0).stop 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 -- 172.21.15.129:0/2228996554 shutdown_connections 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 --2- 172.21.15.129:0/2228996554 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be010e5f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 --2- 172.21.15.129:0/2228996554 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be01036b0 0x7f8be010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 --2- 172.21.15.129:0/2228996554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be0102d90 0x7f8be0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 -- 172.21.15.129:0/2228996554 >> 172.21.15.129:0/2228996554 conn(0x7f8be00fc790 msgr2=0x7f8be00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 -- 172.21.15.129:0/2228996554 shutdown_connections 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.705+0000 7f8be6061640 1 -- 172.21.15.129:0/2228996554 wait complete. 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 Processor -- start 2024-09-05T01:29:17.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 -- start start 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 0x7f8be01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01036b0 0x7f8be0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0111e00 con 0x7f8be010c170 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0111c80 con 0x7f8be01036b0 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8be6061640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f8be0111f80 con 0x7f8be0102d90 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8bdffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53596/0 (socket says 172.21.15.129:53596) 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8bdffff640 1 -- 172.21.15.129:0/1671417179 learned_addr learned my addr 172.21.15.129:0/1671417179 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8bdeffd640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01036b0 0x7f8be0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.706+0000 7f8bdf7fe640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 0x7f8be01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:17.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 -- 172.21.15.129:0/1671417179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 msgr2=0x7f8be01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 0x7f8be01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 -- 172.21.15.129:0/1671417179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01036b0 msgr2=0x7f8be0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01036b0 0x7f8be0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 -- 172.21.15.129:0/1671417179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be01a6700 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdf7fe640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 0x7f8be01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdffff640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802ae70 tx=0x7f8bd802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8034040 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8bd80308f0 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be01a6990 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8004750 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.707+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be01a6e70 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.708+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be0103170 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.709+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8bd8037020 con 0x7f8be010c170 2024-09-05T01:29:17.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.710+0000 7f8bdcff9640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 0x7f8bc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.710+0000 7f8bdf7fe640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 0x7f8bc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.710+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f8bd80f03e0 con 0x7f8be010c170 2024-09-05T01:29:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.710+0000 7f8bdf7fe640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 0x7f8bc80a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00058c0 tx=0x7f8bd0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:17.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.714+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bd80bcf30 con 0x7f8be010c170 2024-09-05T01:29:17.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.920+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8be0107210 con 0x7f8be010c170 2024-09-05T01:29:17.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.921+0000 7f8bdcff9640 1 -- 172.21.15.129:0/1671417179 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f8bd8043e40 con 0x7f8be010c170 2024-09-05T01:29:17.923 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:29:17.923 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:29:17.923 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:29:17.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 msgr2=0x7f8bc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:17.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 0x7f8bc80a5040 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00058c0 tx=0x7f8bd0005850 comp rx=0 tx=0).stop 2024-09-05T01:29:17.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 msgr2=0x7f8be0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:17.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802ae70 tx=0x7f8bd802f5d0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 shutdown_connections 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f8bc80a2b80 0x7f8bc80a5040 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8be010c170 0x7f8be0101470 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8be01036b0 0x7f8be0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 --2- 172.21.15.129:0/1671417179 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8be0102d90 0x7f8be01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 >> 172.21.15.129:0/1671417179 conn(0x7f8be00fc790 msgr2=0x7f8be00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 shutdown_connections 2024-09-05T01:29:17.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:17.923+0000 7f8be6061640 1 -- 172.21.15.129:0/1671417179 wait complete. 2024-09-05T01:29:18.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1671417179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:18.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1671417179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1671417179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:18.681 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id e400dc6a --release --await' 2024-09-05T01:29:18.682 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id e400dc6a --release --await 2024-09-05T01:29:18.930 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:19.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:18 smithi129 ceph-mon[30979]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-09-05T01:29:19.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:18 smithi194 ceph-mon[34941]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-09-05T01:29:19.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:18 smithi183 ceph-mon[34589]: pgmap v773: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 7.7 MiB/s wr, 814 op/s 2024-09-05T01:29:19.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 -- 172.21.15.129:0/626312395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c106f10 msgr2=0x7f250c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 --2- 172.21.15.129:0/626312395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c106f10 0x7f250c10ea90 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009900 tx=0x7f24fc02a410 comp rx=0 tx=0).stop 2024-09-05T01:29:19.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 -- 172.21.15.129:0/626312395 shutdown_connections 2024-09-05T01:29:19.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 --2- 172.21.15.129:0/626312395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c106f10 0x7f250c10ea90 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 --2- 172.21.15.129:0/626312395 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c102790 0x7f250c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 --2- 172.21.15.129:0/626312395 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c101de0 0x7f250c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 -- 172.21.15.129:0/626312395 >> 172.21.15.129:0/626312395 conn(0x7f250c0fd8c0 msgr2=0x7f250c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 -- 172.21.15.129:0/626312395 shutdown_connections 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.490+0000 7f251462f640 1 -- 172.21.15.129:0/626312395 wait complete. 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 Processor -- start 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 -- start start 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 0x7f250c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c106f10 0x7f250c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f250c111e20 con 0x7f250c101de0 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f250c111ca0 con 0x7f250c106f10 2024-09-05T01:29:19.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f251462f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f250c111fa0 con 0x7f250c102790 2024-09-05T01:29:19.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f25123cb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f25123cb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53602/0 (socket says 172.21.15.129:53602) 2024-09-05T01:29:19.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f25123cb640 1 -- 172.21.15.129:0/2712513653 learned_addr learned my addr 172.21.15.129:0/2712513653 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:19.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f2512bcc640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c106f10 0x7f250c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.491+0000 7f2511bca640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 0x7f250c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 -- 172.21.15.129:0/2712513653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 msgr2=0x7f250c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 0x7f250c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 -- 172.21.15.129:0/2712513653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c106f10 msgr2=0x7f250c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c106f10 0x7f250c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 -- 172.21.15.129:0/2712513653 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f250c19d270 con 0x7f250c101de0 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f2511bca640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 0x7f250c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:19.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f25123cb640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f250400cce0 tx=0x7f250400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f250b7fe640 1 -- 172.21.15.129:0/2712513653 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250400f650 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f250c19d560 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f250b7fe640 1 -- 172.21.15.129:0/2712513653 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f250400fca0 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f250b7fe640 1 -- 172.21.15.129:0/2712513653 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504018b90 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.492+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f250c1a4e40 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.494+0000 7f250b7fe640 1 -- 172.21.15.129:0/2712513653 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f250400f7f0 con 0x7f250c101de0 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.494+0000 7f2516a08740 1 Processor -- start 2024-09-05T01:29:19.497 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 -- start start 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb4d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f250b7fe640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 0x7f24f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fab2620 con 0x55fd7fb4d840 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fa59e50 con 0x55fd7fb4cf20 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2516a08740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd7faae5e0 con 0x55fd7fb5c650 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f25123cb640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 0x7f24f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.501 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53606/0 (socket says 172.21.15.129:53606) 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 -- 172.21.15.129:0/2887042558 learned_addr learned my addr 172.21.15.129:0/2887042558 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2512bcc640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f250b7fe640 1 -- 172.21.15.129:0/2712513653 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f25040cf930 con 0x7f250c101de0 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 -- 172.21.15.129:0/2887042558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 msgr2=0x55fd7fb5ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2512bcc640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb4d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 -- 172.21.15.129:0/2887042558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 msgr2=0x55fd7fb4d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f25123cb640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 0x7f24f40a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25040981e0 tx=0x7f2504066950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb4d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2511bca640 1 -- 172.21.15.129:0/2887042558 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd7fa4ad40 con 0x55fd7fb4d840 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.495+0000 7f2512bcc640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:19.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2511bca640 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f250000bcb0 tx=0x7f250000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7416da4141e081 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2508ff9640 1 -- 172.21.15.129:0/2887042558 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500015070 con 0x55fd7fb4d840 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2508ff9640 1 -- 172.21.15.129:0/2887042558 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25000118f0 con 0x55fd7fb4d840 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2508ff9640 1 -- 172.21.15.129:0/2887042558 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500010870 con 0x55fd7fb4d840 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 -- 172.21.15.129:0/2887042558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 msgr2=0x55fd7fb5c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f250000bcb0 tx=0x7f250000f730 comp rx=0 tx=0).stop 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 -- 172.21.15.129:0/2887042558 shutdown_connections 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5c110 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 --2- 172.21.15.129:0/2887042558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb4d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 -- 172.21.15.129:0/2887042558 >> 172.21.15.129:0/2887042558 conn(0x55fd7fb495f0 msgr2=0x55fd7fb499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 -- 172.21.15.129:0/2887042558 shutdown_connections 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.496+0000 7f2516a08740 1 -- 172.21.15.129:0/2887042558 wait complete. 2024-09-05T01:29:19.503 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 -- start start 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 Processor -- start 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4d840 0x55fd7fb59b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fb5f220 con 0x55fd7fb4cf20 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fa3af10 con 0x55fd7fb5c650 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55fd7fb5f3a0 con 0x55fd7fb4d840 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2516a08740 10 client.0 _ll_register_callbacks cb 0x55fd7fa5edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53622/0 (socket says 172.21.15.129:53622) 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 -- 172.21.15.129:0/1920458275 learned_addr learned my addr 172.21.15.129:0/1920458275 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 -- 172.21.15.129:0/1920458275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4d840 msgr2=0x55fd7fb59b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.504 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4d840 0x55fd7fb59b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 -- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb5c650 msgr2=0x55fd7fb5a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.497+0000 7f2512bcc640 1 -- 172.21.15.129:0/1920458275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7f7c1580 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2512bcc640 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f24fc02a930 tx=0x7f24fc0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f2482bdcfb3029 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc035040 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24fc004240 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25000118f0 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24fc004440 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 10 client.16815 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd7f88b960 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25000118f0 con 0x55fd7fb4cf20 2024-09-05T01:29:19.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f24fc004bf0 con 0x55fd7fb4cf20 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 10 client.16815 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 10 client.16815 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 20 client.16815 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fb6c1f0 0x55fd7fb6e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 4 client.16815 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.498+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd7fabfb90 con 0x55fd7fb6c1f0 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.499+0000 7f2511bca640 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fb6c1f0 0x55fd7fb6e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.499+0000 7f24f1ffb640 10 client.16815 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:19.506 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.499+0000 7f2511bca640 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fb6c1f0 0x55fd7fb6e640 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4755b8787c2e4c1a server_cookie=8808dabbda7f83c7 in_seq=0 out_seq=0 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f24f1ffb640 1 -- 172.21.15.129:0/1920458275 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd7fabfb90 con 0x55fd7fb6c1f0 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f24f1ffb640 10 client.16815 handle_command_reply: tid=0 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 client.16815 shutdown 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 msgr2=0x55fd7fb595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f24fc02a930 tx=0x7f24fc0314a0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 shutdown_connections 2024-09-05T01:29:19.507 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fb6c1f0 0x55fd7fb6e640 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb5c650 0x55fd7fb5a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4d840 0x55fd7fb59b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 --2- 172.21.15.129:0/1920458275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb595d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 >> 172.21.15.129:0/1920458275 conn(0x55fd7fb495f0 msgr2=0x55fd7fb4f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f24f1ffb640 0 client.16815 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 shutdown_connections 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 1 -- 172.21.15.129:0/1920458275 wait complete. 2024-09-05T01:29:19.508 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.500+0000 7f2516a08740 20 client.16815 trim_cache size 0 max 16384 2024-09-05T01:29:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.516+0000 7f2516a08740 1 Processor -- start 2024-09-05T01:29:19.518 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 -- start start 2024-09-05T01:29:19.519 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb73f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1af410 con 0x55fd7fb4cf20 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1acce0 con 0x55fd7fb4d840 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2516a08740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1aab30 con 0x55fd7fb5c650 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53638/0 (socket says 172.21.15.129:53638) 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 -- 172.21.15.129:0/235924264 learned_addr learned my addr 172.21.15.129:0/235924264 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2511bca640 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 -- 172.21.15.129:0/235924264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 msgr2=0x55fd7fb73f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb73f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.520 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 -- 172.21.15.129:0/235924264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 msgr2=0x55fd7fb5a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.517+0000 7f2512bcc640 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f2512bcc640 1 -- 172.21.15.129:0/235924264 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd7fb58580 con 0x55fd7fb4cf20 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f2511bca640 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f2512bcc640 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005e40 tx=0x7f24fc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffdc6e67c24cac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f24f3fff640 1 -- 172.21.15.129:0/235924264 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc05a070 con 0x55fd7fb4cf20 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f24f3fff640 1 -- 172.21.15.129:0/235924264 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24fc031e10 con 0x55fd7fb4cf20 2024-09-05T01:29:19.521 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.518+0000 7f24f3fff640 1 -- 172.21.15.129:0/235924264 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc035070 con 0x55fd7fb4cf20 2024-09-05T01:29:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- 172.21.15.129:0/235924264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 msgr2=0x55fd7fb59ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.524 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f24fc005e40 tx=0x7f24fc005b50 comp rx=0 tx=0).stop 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- 172.21.15.129:0/235924264 shutdown_connections 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb5c650 0x55fd7fb73f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb5a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- 172.21.15.129:0/235924264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb59ca0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- 172.21.15.129:0/235924264 >> 172.21.15.129:0/235924264 conn(0x55fd7fb495f0 msgr2=0x55fd7fb59590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- 172.21.15.129:0/235924264 shutdown_connections 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- 172.21.15.129:0/235924264 wait complete. 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- start start 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 Processor -- start 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb60ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.525 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1ab1a0 con 0x55fd7fb5c650 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1ac610 con 0x55fd7fb4d840 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f250c1ab870 con 0x55fd7fb4cf20 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2516a08740 10 client.0 _ll_register_callbacks cb 0x7f24fc032c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.519+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:53646/0 (socket says 172.21.15.129:53646) 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 -- 172.21.15.129:0/4203254413 learned_addr learned my addr 172.21.15.129:0/4203254413 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2511bca640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb60ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 -- 172.21.15.129:0/4203254413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4cf20 msgr2=0x55fd7fb60ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb60ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 -- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 msgr2=0x55fd7fb61000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb61000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.526 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 -- 172.21.15.129:0/4203254413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd7fbd6420 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2511bca640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb61000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2512bcc640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f24fc058040 tx=0x7f24fc0037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c2ec52d149846 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f24f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24fc05a070 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f24fc004fc0 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24fc031e10 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24fc035040 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2516a08740 10 client.16821 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.520+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fd7fbd76d0 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24fc031e10 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f24fc03d450 con 0x55fd7fb5c650 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 10 client.16821 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 10 client.16821 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:19.527 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 20 client.16821 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fbd7d30 0x55fd7fbda120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 4 client.16821 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e400dc6a", "release": true, "await": true}] 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e400dc6a", "release": true, "await": true}) -- 0x55fd7faba0a0 con 0x55fd7fbd7d30 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.521+0000 7f2511bca640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fbd7d30 0x55fd7fbda120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.522+0000 7f2511bca640 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fbd7d30 0x55fd7fbda120 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fdafeca05b00ad6 server_cookie=e7cc984b33162695 in_seq=0 out_seq=0 2024-09-05T01:29:19.528 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.522+0000 7f24f1ffb640 10 client.16821 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.602+0000 7f24f1ffb640 1 -- 172.21.15.129:0/4203254413 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fd7faba0a0 con 0x55fd7fbd7d30 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.602+0000 7f24f1ffb640 10 client.16821 handle_command_reply: tid=0 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.602+0000 7f2516a08740 1 client.16821 shutdown 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 msgr2=0x55fd7fbd5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f24fc058040 tx=0x7f24fc0037e0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 shutdown_connections 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55fd7fbd7d30 0x55fd7fbda120 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fd7fb5c650 0x55fd7fbd5ee0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fd7fb4d840 0x55fd7fb61000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 --2- 172.21.15.129:0/4203254413 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55fd7fb4cf20 0x55fd7fb60ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f24f1ffb640 0 client.16821 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:19.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.603+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 >> 172.21.15.129:0/4203254413 conn(0x55fd7fb495f0 msgr2=0x55fd7fb59db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.604+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 shutdown_connections 2024-09-05T01:29:19.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.604+0000 7f2516a08740 1 -- 172.21.15.129:0/4203254413 wait complete. 2024-09-05T01:29:19.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.606+0000 7f2516a08740 20 client.16821 trim_cache size 0 max 16384 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 52, 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "e400dc6a": { 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 52, 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:29:19.614 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 130.0, 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:19.615 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:19.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:19.616 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:29:19.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 msgr2=0x7f24f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 0x7f24f40a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f25040981e0 tx=0x7f2504066950 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 msgr2=0x7f250c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f250400cce0 tx=0x7f250400a620 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 shutdown_connections 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f24f40a2b80 0x7f24f40a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f250c106f10 0x7f250c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f250c102790 0x7f250c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 --2- 172.21.15.129:0/2712513653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f250c101de0 0x7f250c1982c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 >> 172.21.15.129:0/2712513653 conn(0x7f250c0fd8c0 msgr2=0x7f250c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 shutdown_connections 2024-09-05T01:29:19.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:19.612+0000 7f251462f640 1 -- 172.21.15.129:0/2712513653 wait complete. 2024-09-05T01:29:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 735.0 s, 175.8 tps, lat 555.827 ms stddev 1295.862 2024-09-05T01:29:20.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:20 smithi129 ceph-mon[30979]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-09-05T01:29:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:20 smithi183 ceph-mon[34589]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-09-05T01:29:20.476 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e400dc6a', total seconds elapsed: 13.6 2024-09-05T01:29:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:20 smithi194 ceph-mon[34941]: pgmap v774: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 6.2 MiB/s wr, 651 op/s 2024-09-05T01:29:21.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:21 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:29:21.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:21 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:29:21.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:21 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:29:22.643 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:29:22.643 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:29:22.643 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:22 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:29:22.643 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:22 smithi129 ceph-mon[30979]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 610 op/s 2024-09-05T01:29:22.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:29:22.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:29:22.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:22 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:29:22.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:22 smithi183 ceph-mon[34589]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 610 op/s 2024-09-05T01:29:22.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:29:22.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:29:22.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:22 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:29:22.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:22 smithi194 ceph-mon[34941]: pgmap v775: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 610 op/s 2024-09-05T01:29:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:24 smithi183 ceph-mon[34589]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-09-05T01:29:24.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:24 smithi129 ceph-mon[30979]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-09-05T01:29:24.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:24 smithi194 ceph-mon[34941]: pgmap v776: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.5 MiB/s wr, 587 op/s 2024-09-05T01:29:25.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 740.0 s, 801.9 tps, lat 127.346 ms stddev 61.809 2024-09-05T01:29:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:26 smithi183 ceph-mon[34589]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-09-05T01:29:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:26 smithi129 ceph-mon[30979]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-09-05T01:29:26.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:26 smithi194 ceph-mon[34941]: pgmap v777: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 4.5 MiB/s wr, 493 op/s 2024-09-05T01:29:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:28 smithi129 ceph-mon[30979]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-05T01:29:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:28 smithi194 ceph-mon[34941]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-05T01:29:28.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:28 smithi183 ceph-mon[34589]: pgmap v778: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 599 op/s 2024-09-05T01:29:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 -- 172.21.15.129:0/1535524664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06f190 msgr2=0x7fd56c06fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 --2- 172.21.15.129:0/1535524664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06f190 0x7fd56c06fa30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd564009900 tx=0x7fd56402a400 comp rx=0 tx=0).stop 2024-09-05T01:29:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 -- 172.21.15.129:0/1535524664 shutdown_connections 2024-09-05T01:29:29.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 --2- 172.21.15.129:0/1535524664 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06f190 0x7fd56c06fa30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 --2- 172.21.15.129:0/1535524664 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06e7f0 0x7fd56c06ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 --2- 172.21.15.129:0/1535524664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd56c06d5b0 0x7fd56c06d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 -- 172.21.15.129:0/1535524664 >> 172.21.15.129:0/1535524664 conn(0x7fd56c06bda0 msgr2=0x7fd56c06c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 -- 172.21.15.129:0/1535524664 shutdown_connections 2024-09-05T01:29:29.056 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.054+0000 7fd57339f640 1 -- 172.21.15.129:0/1535524664 wait complete. 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.055+0000 7fd57339f640 1 Processor -- start 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 -- start start 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06e7f0 0x7fd56c1a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd56c06f190 0x7fd56c1adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c11c2a0 con 0x7fd56c06f190 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c11c120 con 0x7fd56c06d5b0 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57339f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c11c420 con 0x7fd56c06e7f0 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.058 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42410/0 (socket says 172.21.15.129:42410) 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 -- 172.21.15.129:0/4180094556 learned_addr learned my addr 172.21.15.129:0/4180094556 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd571b9c640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06e7f0 0x7fd56c1a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 -- 172.21.15.129:0/4180094556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06e7f0 msgr2=0x7fd56c1a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06e7f0 0x7fd56c1a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 -- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd56c06f190 msgr2=0x7fd56c1adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd56c06f190 0x7fd56c1adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.056+0000 7fd57239d640 1 -- 172.21.15.129:0/4180094556 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c1ae570 con 0x7fd56c06d5b0 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd57239d640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00ebc0 tx=0x7fd55c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.059 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd56b7fe640 1 -- 172.21.15.129:0/4180094556 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c010630 con 0x7fd56c06d5b0 2024-09-05T01:29:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd56b7fe640 1 -- 172.21.15.129:0/4180094556 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55c010c80 con 0x7fd56c06d5b0 2024-09-05T01:29:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c1ae860 con 0x7fd56c06d5b0 2024-09-05T01:29:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd56b7fe640 1 -- 172.21.15.129:0/4180094556 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c018ac0 con 0x7fd56c06d5b0 2024-09-05T01:29:29.060 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.057+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c1aee00 con 0x7fd56c06d5b0 2024-09-05T01:29:29.061 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.061+0000 7fd56b7fe640 1 -- 172.21.15.129:0/4180094556 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fd55c018c60 con 0x7fd56c06d5b0 2024-09-05T01:29:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.062+0000 7fd577475740 1 Processor -- start 2024-09-05T01:29:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.062+0000 7fd577475740 1 -- start start 2024-09-05T01:29:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.062+0000 7fd577475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.062+0000 7fd577475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.063 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd56b7fe640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 0x7fd5540a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd56b7fe640 1 -- 172.21.15.129:0/4180094556 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd55c0cfcf0 con 0x7fd56c06d5b0 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd571b9c640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 0x7fd5540a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd577475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a20a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd57239d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.063+0000 7fd57239d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42422/0 (socket says 172.21.15.129:42422) 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd57239d640 1 -- 172.21.15.129:0/943798585 learned_addr learned my addr 172.21.15.129:0/943798585 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd572b9e640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.065 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b9197c2f0 con 0x563b91a1e540 2024-09-05T01:29:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b91924270 con 0x563b91a194b0 2024-09-05T01:29:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b919782b0 con 0x563b91a18b70 2024-09-05T01:29:29.066 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.064+0000 7fd571b9c640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 0x7fd5540a5100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd56c1a7e60 tx=0x7fd56000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd57239d640 1 -- 172.21.15.129:0/943798585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 msgr2=0x563b91a18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd57239d640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd57239d640 1 -- 172.21.15.129:0/943798585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 msgr2=0x563b91a20a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd572b9e640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a20a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd57239d640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a20a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd57239d640 1 -- 172.21.15.129:0/943798585 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b91923340 con 0x563b91a194b0 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.065+0000 7fd572b9e640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a20a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.066+0000 7fd57239d640 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd55c09c0d0 tx=0x7fd55c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8eb773f523297d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.066+0000 7fd568bfe640 1 -- 172.21.15.129:0/943798585 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c09c140 con 0x563b91a194b0 2024-09-05T01:29:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.066+0000 7fd568bfe640 1 -- 172.21.15.129:0/943798585 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55c0a04f0 con 0x563b91a194b0 2024-09-05T01:29:29.068 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.066+0000 7fd568bfe640 1 -- 172.21.15.129:0/943798585 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0987b0 con 0x563b91a194b0 2024-09-05T01:29:29.069 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.069+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 msgr2=0x563b91a1df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.070 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.069+0000 7fd577475740 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd55c09c0d0 tx=0x7fd55c018420 comp rx=0 tx=0).stop 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 shutdown_connections 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a20a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a1df70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 --2- 172.21.15.129:0/943798585 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 >> 172.21.15.129:0/943798585 conn(0x563b91a141e0 msgr2=0x563b91a145e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.071 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.070+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 shutdown_connections 2024-09-05T01:29:29.072 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 -- 172.21.15.129:0/943798585 wait complete. 2024-09-05T01:29:29.073 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 -- start start 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 Processor -- start 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.072+0000 7fd577475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a2a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd577475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b91a214b0 con 0x563b91a1e540 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd577475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b91905290 con 0x563b91a194b0 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd577475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b91a21630 con 0x563b91a18b70 2024-09-05T01:29:29.074 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd577475740 10 client.0 _ll_register_callbacks cb 0x563b91936140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd572b9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a2a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd571b9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.075 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:34812/0 (socket says 172.21.15.129:34812) 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 -- 172.21.15.129:0/703659409 learned_addr learned my addr 172.21.15.129:0/703659409 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 -- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 msgr2=0x563b91a23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a23360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 -- 172.21.15.129:0/703659409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 msgr2=0x563b91a2a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a2a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd57239d640 1 -- 172.21.15.129:0/703659409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b9168f210 con 0x563b91a18b70 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.073+0000 7fd572b9e640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a2a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd57239d640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0cd750 tx=0x7fd55c005ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55705dc5329fc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c0b4070 con 0x563b91a18b70 2024-09-05T01:29:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55c0a04f0 con 0x563b91a18b70 2024-09-05T01:29:29.077 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd55c0107d0 con 0x563b91a18b70 2024-09-05T01:29:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd55c0109d0 con 0x563b91a18b70 2024-09-05T01:29:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd577475740 10 client.26257 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:29.078 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.074+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563b9175b620 con 0x563b91a18b70 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55c0a04f0 con 0x563b91a18b70 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd55c09b0e0 con 0x563b91a18b70 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 10 client.26257 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 20 client.26257 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a2e9a0 0x563b91a30dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 4 client.26257 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b91989a20 con 0x563b91a2e9a0 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.075+0000 7fd571b9c640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a2e9a0 0x563b91a30dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.076+0000 7fd551ffb640 10 client.26257 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.076+0000 7fd571b9c640 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a2e9a0 0x563b91a30dc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4805b81bc17ebc server_cookie=3374c3fea00c9b25 in_seq=0 out_seq=0 2024-09-05T01:29:29.079 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd551ffb640 1 -- 172.21.15.129:0/703659409 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b91989a20 con 0x563b91a2e9a0 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd551ffb640 10 client.26257 handle_command_reply: tid=0 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 client.26257 shutdown 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 msgr2=0x563b91a22e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0cd750 tx=0x7fd55c005ac0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 shutdown_connections 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a2e9a0 0x563b91a30dc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a2a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 --2- 172.21.15.129:0/703659409 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a22e20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 >> 172.21.15.129:0/703659409 conn(0x563b91a141e0 msgr2=0x563b91a159f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.080 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd551ffb640 0 client.26257 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 shutdown_connections 2024-09-05T01:29:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 1 -- 172.21.15.129:0/703659409 wait complete. 2024-09-05T01:29:29.081 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.077+0000 7fd577475740 20 client.26257 trim_cache size 0 max 16384 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 Processor -- start 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 -- start start 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a0eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a2a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1bbec0 con 0x563b91a1e540 2024-09-05T01:29:29.104 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1b9790 con 0x563b91a194b0 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.102+0000 7fd577475740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1b75e0 con 0x563b91a18b70 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46016/0 (socket says 172.21.15.129:46016) 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 -- 172.21.15.129:0/351702873 learned_addr learned my addr 172.21.15.129:0/351702873 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd571b9c640 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a2a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd57239d640 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a0eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 -- 172.21.15.129:0/351702873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 msgr2=0x563b91a0eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a0eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.105 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 -- 172.21.15.129:0/351702873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 msgr2=0x563b91a2a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.103+0000 7fd572b9e640 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a2a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.104+0000 7fd572b9e640 1 -- 172.21.15.129:0/351702873 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b91a19990 con 0x563b91a1e540 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.104+0000 7fd572b9e640 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a9f0 tx=0x7fd564009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5e97ebe71807b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.104+0000 7fd553fff640 1 -- 172.21.15.129:0/351702873 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564035040 con 0x563b91a1e540 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.105+0000 7fd553fff640 1 -- 172.21.15.129:0/351702873 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564041070 con 0x563b91a1e540 2024-09-05T01:29:29.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.105+0000 7fd553fff640 1 -- 172.21.15.129:0/351702873 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640035f0 con 0x563b91a1e540 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 -- 172.21.15.129:0/351702873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 msgr2=0x563b91a12a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a9f0 tx=0x7fd564009260 comp rx=0 tx=0).stop 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 -- 172.21.15.129:0/351702873 shutdown_connections 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a1e540 0x563b91a12a90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a194b0 0x563b91a2a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 --2- 172.21.15.129:0/351702873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a18b70 0x563b91a0eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.106+0000 7fd577475740 1 -- 172.21.15.129:0/351702873 >> 172.21.15.129:0/351702873 conn(0x563b91a141e0 msgr2=0x563b91a1a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.107+0000 7fd577475740 1 -- 172.21.15.129:0/351702873 shutdown_connections 2024-09-05T01:29:29.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.107+0000 7fd577475740 1 -- 172.21.15.129:0/351702873 wait complete. 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.107+0000 7fd577475740 1 -- start start 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.107+0000 7fd577475740 1 Processor -- start 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd577475740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd577475740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 0x563b91a27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd577475740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a1e540 0x563b91a97690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd57239d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd57239d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46024/0 (socket says 172.21.15.129:46024) 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd57239d640 1 -- 172.21.15.129:0/244878924 learned_addr learned my addr 172.21.15.129:0/244878924 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:29.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd571b9c640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 0x563b91a27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1b7c50 con 0x563b91a18b70 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1b90c0 con 0x563b91a1e540 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.108+0000 7fd572b9e640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a1e540 0x563b91a97690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c1b8320 con 0x563b91a194b0 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd577475740 10 client.0 _ll_register_callbacks cb 0x7fd55c018f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 -- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a1e540 msgr2=0x563b91a97690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a1e540 0x563b91a97690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 -- 172.21.15.129:0/244878924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 msgr2=0x563b91a27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b91a27ce0 con 0x563b91a194b0 2024-09-05T01:29:29.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd57239d640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:29:29.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.109+0000 7fd571b9c640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 0x563b91a27670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x563b91a31640 tx=0x7fd560013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5068adf2cf749fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd560020a90 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fd560020c30 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd564041070 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd560037050 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd577475740 10 client.26263 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563b91a98d30 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd564041070 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fd560004940 con 0x563b91a194b0 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.110+0000 7fd577475740 10 client.26263 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:29.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd577475740 10 client.26263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd577475740 20 client.26263 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a99660 0x563b91a9ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd577475740 4 client.26263 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b91983d70 con 0x563b91a99660 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.111+0000 7fd57239d640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a99660 0x563b91a9ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.112+0000 7fd57239d640 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a99660 0x563b91a9ba80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43cb41d29b710a7c server_cookie=74e4dc21e6cec7fa in_seq=0 out_seq=0 2024-09-05T01:29:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.112+0000 7fd551ffb640 10 client.26263 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.140+0000 7fd551ffb640 1 -- 172.21.15.129:0/244878924 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b91983d70 con 0x563b91a99660 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.140+0000 7fd551ffb640 10 client.26263 handle_command_reply: tid=0 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.140+0000 7fd577475740 1 client.26263 shutdown 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 msgr2=0x563b91a27670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 0x563b91a27670 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x563b91a31640 tx=0x7fd560013000 comp rx=0 tx=0).stop 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 shutdown_connections 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b91a99660 0x563b91a9ba80 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b91a1e540 0x563b91a97690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b91a194b0 0x563b91a27670 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 --2- 172.21.15.129:0/244878924 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b91a18b70 0x563b91a27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 >> 172.21.15.129:0/244878924 conn(0x563b91a141e0 msgr2=0x563b91a15090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd551ffb640 0 client.26263 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 shutdown_connections 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 1 -- 172.21.15.129:0/244878924 wait complete. 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.141+0000 7fd577475740 20 client.26263 trim_cache size 0 max 16384 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:29:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:29:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:29:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:29:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.142+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 msgr2=0x7fd5540a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.142+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 0x7fd5540a5100 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd56c1a7e60 tx=0x7fd56000b390 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.142+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 msgr2=0x7fd56c1a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.142+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd55c00ebc0 tx=0x7fd55c00c6a0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.142+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 shutdown_connections 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fd5540a2c40 0x7fd5540a5100 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd56c06f190 0x7fd56c1adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fd56c06e7f0 0x7fd56c1a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 --2- 172.21.15.129:0/4180094556 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd56c06d5b0 0x7fd56c1a69c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 >> 172.21.15.129:0/4180094556 conn(0x7fd56c06bda0 msgr2=0x7fd56c10b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 shutdown_connections 2024-09-05T01:29:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:29.143+0000 7fd57339f640 1 -- 172.21.15.129:0/4180094556 wait complete. 2024-09-05T01:29:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 745.0 s, 770.7 tps, lat 129.400 ms stddev 76.582 2024-09-05T01:29:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:30 smithi183 ceph-mon[34589]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-09-05T01:29:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:30 smithi129 ceph-mon[30979]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-09-05T01:29:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:30 smithi194 ceph-mon[34941]: pgmap v779: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.1 MiB/s wr, 559 op/s 2024-09-05T01:29:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:32 smithi183 ceph-mon[34589]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-09-05T01:29:32.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:32 smithi129 ceph-mon[30979]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-09-05T01:29:32.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:32 smithi194 ceph-mon[34941]: pgmap v780: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 674 op/s 2024-09-05T01:29:34.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:34 smithi183 ceph-mon[34589]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-05T01:29:34.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:34 smithi129 ceph-mon[30979]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-05T01:29:34.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:34 smithi194 ceph-mon[34941]: pgmap v781: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.8 MiB/s wr, 757 op/s 2024-09-05T01:29:35.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 750.0 s, 826.6 tps, lat 121.340 ms stddev 62.175 2024-09-05T01:29:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:36 smithi129 ceph-mon[30979]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 700 op/s 2024-09-05T01:29:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:36 smithi194 ceph-mon[34941]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 700 op/s 2024-09-05T01:29:36.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:36 smithi183 ceph-mon[34589]: pgmap v782: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.3 MiB/s wr, 700 op/s 2024-09-05T01:29:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:38 smithi183 ceph-mon[34589]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 838 op/s 2024-09-05T01:29:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:38 smithi129 ceph-mon[30979]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 838 op/s 2024-09-05T01:29:38.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:38 smithi194 ceph-mon[34941]: pgmap v783: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 838 op/s 2024-09-05T01:29:39.049 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.8 2024-09-05T01:29:39.049 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:29:39.542 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:40.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.138+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2691969839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 msgr2=0x7ffbb8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:40.139 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.138+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2691969839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb8110250 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009860 tx=0x7ffba802a380 comp rx=0 tx=0).stop 2024-09-05T01:29:40.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.139+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2691969839 shutdown_connections 2024-09-05T01:29:40.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.139+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2691969839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.139+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2691969839 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.139+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2691969839 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb8110250 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.140 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.139+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2691969839 >> 172.21.15.129:0/2691969839 conn(0x7ffbb806e140 msgr2=0x7ffbb806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:40.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.140+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2691969839 shutdown_connections 2024-09-05T01:29:40.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.140+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2691969839 wait complete. 2024-09-05T01:29:40.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 Processor -- start 2024-09-05T01:29:40.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 -- start start 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb811a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8116890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb8116dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.141+0000 7ffbbe2ea640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113180 con 0x7ffbb811b140 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbbe2ea640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113000 con 0x7ffbb8109830 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbbe2ea640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb8113300 con 0x7ffbb8109450 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbbc887640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb8116dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbb77fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8116890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbbc887640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb8116dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:41156/0 (socket says 172.21.15.129:41156) 2024-09-05T01:29:40.143 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbbc887640 1 -- 172.21.15.129:0/2154516220 learned_addr learned my addr 172.21.15.129:0/2154516220 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:40.144 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.142+0000 7ffbb7fff640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb811a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 -- 172.21.15.129:0/2154516220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 msgr2=0x7ffbb811a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb811a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 -- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 msgr2=0x7ffbb8116dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb8116dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 -- 172.21.15.129:0/2154516220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb81174b0 con 0x7ffbb8109830 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb77fe640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8116890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbac01f9b0 tx=0x7ffbac01fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac01db90 con 0x7ffbb8109830 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffbac01dd30 con 0x7ffbb8109830 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac029780 con 0x7ffbb8109830 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.143+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb81ab170 con 0x7ffbb8109830 2024-09-05T01:29:40.145 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.144+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb81ab510 con 0x7ffbb8109830 2024-09-05T01:29:40.146 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.145+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb8073b00 con 0x7ffbb8109830 2024-09-05T01:29:40.149 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.148+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffbac004020 con 0x7ffbb8109830 2024-09-05T01:29:40.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.149+0000 7ffbb57fa640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 0x7ffba00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:40.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.149+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffbac0e0ba0 con 0x7ffbb8109830 2024-09-05T01:29:40.150 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.149+0000 7ffbb7fff640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 0x7ffba00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:40.151 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.150+0000 7ffbb7fff640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 0x7ffba00a5140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffba80057c0 tx=0x7ffba8005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:40.152 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.152+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbac037070 con 0x7ffbb8109830 2024-09-05T01:29:40.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 755.0 s, 823.4 tps, lat 120.823 ms stddev 52.528 2024-09-05T01:29:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:40 smithi129 ceph-mon[30979]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 763 op/s 2024-09-05T01:29:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:40 smithi194 ceph-mon[34941]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 763 op/s 2024-09-05T01:29:40.545 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.544+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb8117dc0 con 0x7ffbb8109830 2024-09-05T01:29:40.546 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.545+0000 7ffbb57fa640 1 -- 172.21.15.129:0/2154516220 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7ffbac0ad690 con 0x7ffbb8109830 2024-09-05T01:29:40.546 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:29:40.547 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:29:40.547 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:29:40.550 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.549+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 msgr2=0x7ffba00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.549+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 0x7ffba00a5140 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffba80057c0 tx=0x7ffba8005c90 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.549+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 msgr2=0x7ffbb8116890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.549+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8116890 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffbac01f9b0 tx=0x7ffbac01fe70 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 shutdown_connections 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffba00a2c80 0x7ffba00a5140 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbb811b140 0x7ffbb8116dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffbb8109830 0x7ffbb8116890 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 --2- 172.21.15.129:0/2154516220 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffbb8109450 0x7ffbb811a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:40.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 >> 172.21.15.129:0/2154516220 conn(0x7ffbb806e140 msgr2=0x7ffbb8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.550+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 shutdown_connections 2024-09-05T01:29:40.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:40.551+0000 7ffbbe2ea640 1 -- 172.21.15.129:0/2154516220 wait complete. 2024-09-05T01:29:40.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:40 smithi183 ceph-mon[34589]: pgmap v784: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.0 MiB/s wr, 763 op/s 2024-09-05T01:29:41.266 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 143.8 --await-for 45.0' 2024-09-05T01:29:41.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 143.8 --await-for 45.0 2024-09-05T01:29:41.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:41 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2154516220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:41.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:41 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2154516220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:41.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:41 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2154516220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:29:41.718 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.273+0000 7fe6b12d7640 1 -- 172.21.15.129:0/2836577197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac1084c0 msgr2=0x7fe6ac1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.273+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/2836577197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac1088c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009930 tx=0x7fe69c02a420 comp rx=0 tx=0).stop 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 -- 172.21.15.129:0/2836577197 shutdown_connections 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/2836577197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/2836577197 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/2836577197 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac1088c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 -- 172.21.15.129:0/2836577197 >> 172.21.15.129:0/2836577197 conn(0x7fe6ac06b6d0 msgr2=0x7fe6ac079830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 -- 172.21.15.129:0/2836577197 shutdown_connections 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.274+0000 7fe6b12d7640 1 -- 172.21.15.129:0/2836577197 wait complete. 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.275+0000 7fe6b12d7640 1 Processor -- start 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.275+0000 7fe6b12d7640 1 -- start start 2024-09-05T01:29:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac0806a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac080be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac081c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac07aef0 con 0x7fe6ac1084c0 2024-09-05T01:29:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac07ad70 con 0x7fe6ac109cd0 2024-09-05T01:29:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6b12d7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac07b070 con 0x7fe6ac109310 2024-09-05T01:29:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac080be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac081c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac081c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56196/0 (socket says 172.21.15.129:56196) 2024-09-05T01:29:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.276+0000 7fe6ab7fe640 1 -- 172.21.15.129:0/771072474 learned_addr learned my addr 172.21.15.129:0/771072474 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac0806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/771072474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 msgr2=0x7fe6ac081c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac081c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac1084c0 msgr2=0x7fe6ac0806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac0806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/771072474 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac082290 con 0x7fe6ac109310 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac080be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9f0 tx=0x7fe6a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe693fff640 1 -- 172.21.15.129:0/771072474 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a000ccc0 con 0x7fe6ac109310 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe693fff640 1 -- 172.21.15.129:0/771072474 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a0005020 con 0x7fe6ac109310 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.277+0000 7fe693fff640 1 -- 172.21.15.129:0/771072474 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0004a00 con 0x7fe6ac109310 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.278+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac082580 con 0x7fe6ac109310 2024-09-05T01:29:42.279 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.278+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac082a80 con 0x7fe6ac109310 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.281+0000 7fe6b4070740 1 Processor -- start 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.281+0000 7fe6b4070740 1 -- start start 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.281+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.281+0000 7fe693fff640 1 -- 172.21.15.129:0/771072474 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe6a0004ba0 con 0x7fe6ac109310 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.282 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6b4070740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d03e8272f0 con 0x55d03e8c39a0 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6b4070740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d03e7cf270 con 0x55d03e8c42e0 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6b4070740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d03e8232b0 con 0x55d03e8c9370 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.283 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42442/0 (socket says 172.21.15.129:42442) 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6ab7fe640 1 -- 172.21.15.129:0/97039091 learned_addr learned my addr 172.21.15.129:0/97039091 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.282+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.283+0000 7fe693fff640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.283+0000 7fe6ab7fe640 1 -- 172.21.15.129:0/97039091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 msgr2=0x55d03e8cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.283+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.284 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.283+0000 7fe6ab7fe640 1 -- 172.21.15.129:0/97039091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 msgr2=0x55d03e8c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.283+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.284+0000 7fe6ab7fe640 1 -- 172.21.15.129:0/97039091 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03e7ce340 con 0x55d03e8c39a0 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.284+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.284+0000 7fe693fff640 1 -- 172.21.15.129:0/771072474 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe6a00cf3e0 con 0x7fe6ac109310 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.284+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.285 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.284+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4011570 tx=0x7fe6a4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b682c0322a282e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.286 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.286+0000 7fe6917fa640 1 -- 172.21.15.129:0/97039091 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400f880 con 0x55d03e8c39a0 2024-09-05T01:29:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.286+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 0x7fe6940a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe6a401d000 tx=0x7fe6a401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.286+0000 7fe6917fa640 1 -- 172.21.15.129:0/97039091 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a401e070 con 0x55d03e8c39a0 2024-09-05T01:29:42.287 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.286+0000 7fe6917fa640 1 -- 172.21.15.129:0/97039091 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4023c90 con 0x55d03e8c39a0 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.287+0000 7fe6b4070740 1 -- 172.21.15.129:0/97039091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 msgr2=0x55d03e8c3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.287+0000 7fe6b4070740 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4011570 tx=0x7fe6a4011a30 comp rx=0 tx=0).stop 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 -- 172.21.15.129:0/97039091 shutdown_connections 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 --2- 172.21.15.129:0/97039091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8c3da0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 -- 172.21.15.129:0/97039091 >> 172.21.15.129:0/97039091 conn(0x55d03e8bf010 msgr2=0x55d03e8bf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 -- 172.21.15.129:0/97039091 shutdown_connections 2024-09-05T01:29:42.289 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 -- 172.21.15.129:0/97039091 wait complete. 2024-09-05T01:29:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 -- start start 2024-09-05T01:29:42.290 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.288+0000 7fe6b4070740 1 Processor -- start 2024-09-05T01:29:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.292 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d03e8cc2e0 con 0x55d03e8c39a0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d03e7b0290 con 0x55d03e8c42e0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d03e8cc460 con 0x55d03e8c9370 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6b4070740 10 client.0 _ll_register_callbacks cb 0x55d03e7e1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56204/0 (socket says 172.21.15.129:56204) 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2286028405 learned_addr learned my addr 172.21.15.129:0/2286028405 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2286028405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 msgr2=0x55d03e8d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2286028405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 msgr2=0x55d03e8cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.293 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2286028405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03e53a210 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.289+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9c0 tx=0x7fe6a009b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51c3ce14d50fab8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.290+0000 7fe68eff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.290+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a001c070 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.290+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe6a00cf120 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.290+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a401e070 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.290+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6a009b830 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 10 client.26420 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d03e606620 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a009fbe0 con 0x55d03e8c42e0 2024-09-05T01:29:42.294 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe6a0097ea0 con 0x55d03e8c42e0 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 10 client.26420 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 20 client.26420 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e8d96d0 0x55d03e8dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 4 client.26420 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe6a401e070 con 0x55d03e8d96d0 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e8d96d0 0x55d03e8dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.295 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e8d96d0 0x55d03e8dbac0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f5c95f3f1a89e2 server_cookie=5203188f4ab090 in_seq=0 out_seq=0 2024-09-05T01:29:42.296 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fe68eff5640 10 client.26420 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.291+0000 7fed3b7fe640 1 -- 172.21.15.129:0/2165121288 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30031880 con 0x7fed3c106a90 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 -- 172.21.15.129:0/2165121288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c106a90 msgr2=0x7fed3c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 --2- 172.21.15.129:0/2165121288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c106a90 0x7fed3c106e70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fed30009900 tx=0x7fed3002a400 comp rx=0 tx=0).stop 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 -- 172.21.15.129:0/2165121288 shutdown_connections 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 --2- 172.21.15.129:0/2165121288 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c100860 0x7fed3c1045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 --2- 172.21.15.129:0/2165121288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c107440 0x7fed3c100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 --2- 172.21.15.129:0/2165121288 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c106a90 0x7fed3c106e70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.296 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 -- 172.21.15.129:0/2165121288 >> 172.21.15.129:0/2165121288 conn(0x7fed3c077230 msgr2=0x7fed3c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 -- 172.21.15.129:0/2165121288 shutdown_connections 2024-09-05T01:29:42.297 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.292+0000 7fed42be9640 1 -- 172.21.15.129:0/2165121288 wait complete. 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fe68eff5640 1 -- 172.21.15.129:0/2286028405 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe6a401e070 con 0x55d03e8d96d0 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fe68eff5640 10 client.26420 handle_command_reply: tid=0 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fe6b4070740 1 client.26420 shutdown 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 msgr2=0x55d03e8ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9c0 tx=0x7fe6a009b6c0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 shutdown_connections 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e8d96d0 0x55d03e8dbac0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe68eff5640 0 client.26420 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.298 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8ce010 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2286028405 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 >> 172.21.15.129:0/2286028405 conn(0x55d03e8bf010 msgr2=0x55d03e8c0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 shutdown_connections 2024-09-05T01:29:42.299 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 1 -- 172.21.15.129:0/2286028405 wait complete. 2024-09-05T01:29:42.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fed42be9640 1 Processor -- start 2024-09-05T01:29:42.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.293+0000 7fed42be9640 1 -- start start 2024-09-05T01:29:42.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 0x7fed3c1a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.299 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 0x7fed3c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c19e7c0 0x7fed3c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c118490 con 0x7fed3c106a90 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c118310 con 0x7fed3c100860 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed42be9640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c118610 con 0x7fed3c19e7c0 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 0x7fed3c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 0x7fed3c1a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42478/0 (socket says 172.21.15.129:42478) 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed3bfff640 1 -- 172.21.15.129:0/4030683776 learned_addr learned my addr 172.21.15.129:0/4030683776 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.300 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed40985640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 0x7fed3c1a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.301 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fe6b4070740 20 client.26420 trim_cache size 0 max 16384 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.294+0000 7fed41186640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c19e7c0 0x7fed3c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 -- 172.21.15.129:0/4030683776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c19e7c0 msgr2=0x7fed3c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c19e7c0 0x7fed3c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 -- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 msgr2=0x7fed3c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 0x7fed3c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 -- 172.21.15.129:0/4030683776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3c19f340 con 0x7fed3c100860 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed40985640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 0x7fed3c1a4640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a9f0 tx=0x7fed30003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed39ffb640 1 -- 172.21.15.129:0/4030683776 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3003f070 con 0x7fed3c100860 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.295+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3c19f5d0 con 0x7fed3c100860 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.296+0000 7fed39ffb640 1 -- 172.21.15.129:0/4030683776 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30031b50 con 0x7fed3c100860 2024-09-05T01:29:42.302 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.296+0000 7fed39ffb640 1 -- 172.21.15.129:0/4030683776 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30035030 con 0x7fed3c100860 2024-09-05T01:29:42.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.296+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed3c06be90 con 0x7fed3c100860 2024-09-05T01:29:42.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.300+0000 7fed44fc2740 1 Processor -- start 2024-09-05T01:29:42.303 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.301+0000 7fed44fc2740 1 -- start start 2024-09-05T01:29:42.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed39ffb640 1 -- 172.21.15.129:0/4030683776 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fed30042070 con 0x7fed3c100860 2024-09-05T01:29:42.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 0x556c6e1b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.304 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e1c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c6e11b2a0 con 0x556c6e1c0940 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c6e0c2b10 con 0x556c6e1b6080 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed44fc2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556c6e117260 con 0x556c6e1b5760 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed40985640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed40985640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56244/0 (socket says 172.21.15.129:56244) 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.302+0000 7fed40985640 1 -- 172.21.15.129:0/3919689890 learned_addr learned my addr 172.21.15.129:0/3919689890 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.303+0000 7fed41186640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 0x556c6e1b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.303+0000 7fed3bfff640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e1c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.303+0000 7fed39ffb640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 0x7fed2c0a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.305 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.303+0000 7fed39ffb640 1 -- 172.21.15.129:0/4030683776 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fed300ef6e0 con 0x7fed3c100860 2024-09-05T01:29:42.306 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 -- 172.21.15.129:0/3919689890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 msgr2=0x556c6e1b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 0x556c6e1b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 -- 172.21.15.129:0/3919689890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 msgr2=0x556c6e1c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e1c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 -- 172.21.15.129:0/3919689890 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c6de8cb90 con 0x556c6e1b6080 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed41186640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 0x556c6e1b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed3bfff640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e1c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed40985640 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed300ecdd0 tx=0x7fed3002a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd509c83797d65b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed41186640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 0x7fed2c0a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed26ffd640 1 -- 172.21.15.129:0/3919689890 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30049070 con 0x556c6e1b6080 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed26ffd640 1 -- 172.21.15.129:0/3919689890 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed300bc230 con 0x556c6e1b6080 2024-09-05T01:29:42.307 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.304+0000 7fed26ffd640 1 -- 172.21.15.129:0/3919689890 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300b9e30 con 0x556c6e1b6080 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed41186640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 0x7fed2c0a51d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fed34005fd0 tx=0x7fed34004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 -- 172.21.15.129:0/3919689890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 msgr2=0x556c6e1c0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fed300ecdd0 tx=0x7fed3002a400 comp rx=0 tx=0).stop 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 -- 172.21.15.129:0/3919689890 shutdown_connections 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e1c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1c0400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 --2- 172.21.15.129:0/3919689890 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b5760 0x556c6e1b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.306+0000 7fed44fc2740 1 -- 172.21.15.129:0/3919689890 >> 172.21.15.129:0/3919689890 conn(0x556c6e1b1cf0 msgr2=0x556c6e1b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.308 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.307+0000 7fed44fc2740 1 -- 172.21.15.129:0/3919689890 shutdown_connections 2024-09-05T01:29:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.307+0000 7fed44fc2740 1 -- 172.21.15.129:0/3919689890 wait complete. 2024-09-05T01:29:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.307+0000 7fed44fc2740 1 -- start start 2024-09-05T01:29:42.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.307+0000 7fed44fc2740 1 Processor -- start 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556c6e1c32d0 con 0x556c6e1b5760 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556c6e0a36e0 con 0x556c6e1b6080 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556c6e1c3450 con 0x556c6e1c0940 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:56946/0 (socket says 172.21.15.129:56946) 2024-09-05T01:29:42.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 -- 172.21.15.129:0/4233223669 learned_addr learned my addr 172.21.15.129:0/4233223669 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed44fc2740 10 client.0 _ll_register_callbacks cb 0x556c6e0c7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 -- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 msgr2=0x556c6e1bddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1bddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 -- 172.21.15.129:0/4233223669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 msgr2=0x556c6e1bd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.308+0000 7fed41186640 1 -- 172.21.15.129:0/4233223669 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c6e1cc730 con 0x556c6e1c0940 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed41186640 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed34012930 tx=0x7fed34004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d877ad01d07504 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.311 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:29:42.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3401dd70 con 0x556c6e1c0940 2024-09-05T01:29:42.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fed3401e450 con 0x556c6e1c0940 2024-09-05T01:29:42.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed300bc230 con 0x556c6e1c0940 2024-09-05T01:29:42.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed3401e650 con 0x556c6e1c0940 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed44fc2740 10 client.26293 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.309+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556c6e1cf070 con 0x556c6e1c0940 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed300bc230 con 0x556c6e1c0940 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fed3400f040 con 0x556c6e1c0940 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 10 client.26293 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 10 client.26293 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 20 client.26293 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e1d0580 0x556c6e1d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 4 client.26293 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c6e1289d0 con 0x556c6e1d0580 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.310+0000 7fed40985640 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e1d0580 0x556c6e1d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.311+0000 7fed40985640 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e1d0580 0x556c6e1d2a10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965425f703c2c424 server_cookie=36b6643125ff063b in_seq=0 out_seq=0 2024-09-05T01:29:42.314 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.311+0000 7fed1bfff640 10 client.26293 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.311+0000 7fed1bfff640 1 -- 172.21.15.129:0/4233223669 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c6e1289d0 con 0x556c6e1d0580 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.311+0000 7fed1bfff640 10 client.26293 handle_command_reply: tid=0 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 client.26293 shutdown 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 msgr2=0x556c6e1be330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fed34012930 tx=0x7fed34004ab0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 shutdown_connections 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e1d0580 0x556c6e1d2a10 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed1bfff640 0 client.26293 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1c0940 0x556c6e1be330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b6080 0x556c6e1bddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 --2- 172.21.15.129:0/4233223669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.312+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 >> 172.21.15.129:0/4233223669 conn(0x556c6e1b1cf0 msgr2=0x556c6e1b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.313+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 shutdown_connections 2024-09-05T01:29:42.315 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.313+0000 7fed44fc2740 1 -- 172.21.15.129:0/4233223669 wait complete. 2024-09-05T01:29:42.316 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.313+0000 7fed44fc2740 20 client.26293 trim_cache size 0 max 16384 2024-09-05T01:29:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.320+0000 7fe6b4070740 1 Processor -- start 2024-09-05T01:29:42.321 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 -- start start 2024-09-05T01:29:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 0x55d03e8be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 0x55d03e8d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.322 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c9370 0x55d03e8e0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1bbf70 con 0x55d03e8c9370 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1b9840 con 0x55d03e8c39a0 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6b4070740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1b7690 con 0x55d03e8c42e0 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6aaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 0x55d03e8be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6aaffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 0x55d03e8be0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56260/0 (socket says 172.21.15.129:56260) 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6aaffd640 1 -- 172.21.15.129:0/2615174783 learned_addr learned my addr 172.21.15.129:0/2615174783 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.321+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 0x55d03e8d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.322+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2615174783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 msgr2=0x55d03e8be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.322+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 0x55d03e8be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.322+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2615174783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c9370 msgr2=0x55d03e8e0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.322+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c9370 0x55d03e8e0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.323 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.322+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2615174783 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03e8c8710 con 0x55d03e8c42e0 2024-09-05T01:29:42.324 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.323+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 0x55d03e8d5010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00862b0 tx=0x7fe6a009bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9a26f41a8182f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.324+0000 7fe690ff9640 1 -- 172.21.15.129:0/2615174783 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a001c070 con 0x55d03e8c42e0 2024-09-05T01:29:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.325+0000 7fe690ff9640 1 -- 172.21.15.129:0/2615174783 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a00a44a0 con 0x55d03e8c42e0 2024-09-05T01:29:42.325 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.325+0000 7fe690ff9640 1 -- 172.21.15.129:0/2615174783 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0097ea0 con 0x55d03e8c42e0 2024-09-05T01:29:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 -- 172.21.15.129:0/2615174783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 msgr2=0x55d03e8d5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 0x55d03e8d5010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00862b0 tx=0x7fe6a009bae0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 -- 172.21.15.129:0/2615174783 shutdown_connections 2024-09-05T01:29:42.327 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c9370 0x55d03e8e0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c42e0 0x55d03e8d5010 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2615174783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c39a0 0x55d03e8be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.326+0000 7fe6b4070740 1 -- 172.21.15.129:0/2615174783 >> 172.21.15.129:0/2615174783 conn(0x55d03e8bf010 msgr2=0x55d03e8bd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.328 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.328+0000 7fe6b4070740 1 -- 172.21.15.129:0/2615174783 shutdown_connections 2024-09-05T01:29:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.328+0000 7fe6b4070740 1 -- 172.21.15.129:0/2615174783 wait complete. 2024-09-05T01:29:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 -- start start 2024-09-05T01:29:42.329 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 Processor -- start 2024-09-05T01:29:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.330 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1b7d00 con 0x55d03e8c39a0 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1b9170 con 0x55d03e8c42e0 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1b83d0 con 0x55d03e8c9370 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.329+0000 7fe6b4070740 10 client.0 _ll_register_callbacks cb 0x7fe6a0004e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.330+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.330+0000 7fe6aa7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56274/0 (socket says 172.21.15.129:56274) 2024-09-05T01:29:42.331 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.330+0000 7fe6aa7fc640 1 -- 172.21.15.129:0/2037935972 learned_addr learned my addr 172.21.15.129:0/2037935972 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.330+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 -- 172.21.15.129:0/2037935972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 msgr2=0x55d03e8d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 -- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 msgr2=0x55d03e8d32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.332 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 -- 172.21.15.129:0/2037935972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03e8d4900 con 0x55d03e8c39a0 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6ab7fe640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.331+0000 7fe6aaffd640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8d2da0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe69c031d70 tx=0x7fe69c031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0253169f33449b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.332+0000 7fe68eff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:42.333 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.332+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69c048070 con 0x55d03e8c39a0 2024-09-05T01:29:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.332+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fe69c004250 con 0x55d03e8c39a0 2024-09-05T01:29:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.332+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6a00a44a0 con 0x55d03e8c39a0 2024-09-05T01:29:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.333+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe69c054050 con 0x55d03e8c39a0 2024-09-05T01:29:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.333+0000 7fe6b4070740 10 client.16857 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:42.334 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.333+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d03e944af0 con 0x55d03e8c39a0 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.333+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe6a00a44a0 con 0x55d03e8c39a0 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fe69c004880 con 0x55d03e8c39a0 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 10 client.16857 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 20 client.16857 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e945450 0x55d03e947870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 4 client.16857 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d03e834a20 con 0x55d03e945450 2024-09-05T01:29:42.335 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.334+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e945450 0x55d03e947870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.335+0000 7fe68eff5640 10 client.16857 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.336 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.335+0000 7fe6aa7fc640 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e945450 0x55d03e947870 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b14565e9c34f63 server_cookie=255daa98bb600335 in_seq=0 out_seq=0 2024-09-05T01:29:42.346 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.345+0000 7fed44fc2740 1 Processor -- start 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 -- start start 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1d8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1c0940 0x556c6e1bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1c04e0 con 0x556c6e1b5760 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1bddb0 con 0x556c6e1c0940 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed44fc2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c106e70 con 0x556c6e1b6080 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed40985640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed40985640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42520/0 (socket says 172.21.15.129:42520) 2024-09-05T01:29:42.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed40985640 1 -- 172.21.15.129:0/2826012050 learned_addr learned my addr 172.21.15.129:0/2826012050 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed3bfff640 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1d8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.346+0000 7fed41186640 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1c0940 0x556c6e1bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 -- 172.21.15.129:0/2826012050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 msgr2=0x556c6e1d8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1d8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 -- 172.21.15.129:0/2826012050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1c0940 msgr2=0x556c6e1bab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1c0940 0x556c6e1bab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 -- 172.21.15.129:0/2826012050 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c6e1c3170 con 0x556c6e1b5760 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed40985640 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fed300bb830 tx=0x7fed30031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84de71e2bc9626ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed267fc640 1 -- 172.21.15.129:0/2826012050 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300bce60 con 0x556c6e1b5760 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed267fc640 1 -- 172.21.15.129:0/2826012050 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556c6e1289d0 con 0x556c6e1b5760 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.347+0000 7fed267fc640 1 -- 172.21.15.129:0/2826012050 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30049040 con 0x556c6e1b5760 2024-09-05T01:29:42.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.348+0000 7fed44fc2740 1 -- 172.21.15.129:0/2826012050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 msgr2=0x556c6e1d82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.348+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fed300bb830 tx=0x7fed30031400 comp rx=0 tx=0).stop 2024-09-05T01:29:42.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 -- 172.21.15.129:0/2826012050 shutdown_connections 2024-09-05T01:29:42.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1c0940 0x556c6e1bab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1d8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2826012050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1b5760 0x556c6e1d82e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 -- 172.21.15.129:0/2826012050 >> 172.21.15.129:0/2826012050 conn(0x556c6e1b1cf0 msgr2=0x556c6e1bc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.349+0000 7fed44fc2740 1 -- 172.21.15.129:0/2826012050 shutdown_connections 2024-09-05T01:29:42.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.350+0000 7fed44fc2740 1 -- 172.21.15.129:0/2826012050 wait complete. 2024-09-05T01:29:42.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.350+0000 7fed44fc2740 1 -- start start 2024-09-05T01:29:42.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.350+0000 7fed44fc2740 1 Processor -- start 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 0x556c6e1c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e23a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:56968/0 (socket says 172.21.15.129:56968) 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed40985640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 0x556c6e1c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed40985640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 0x556c6e1c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:56298/0 (socket says 172.21.15.129:56298) 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed41186640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e23a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed41186640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e23a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42534/0 (socket says 172.21.15.129:42534) 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed41186640 1 -- 172.21.15.129:0/2648312310 learned_addr learned my addr 172.21.15.129:0/2648312310 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c074260 con 0x556c6e1c0940 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1bd6e0 con 0x556c6e1b5760 2024-09-05T01:29:42.353 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fed3c1bc940 con 0x556c6e1b6080 2024-09-05T01:29:42.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.351+0000 7fed44fc2740 10 client.0 _ll_register_callbacks cb 0x7fed34012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:42.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 -- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 msgr2=0x556c6e1c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 0x556c6e1c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 -- 172.21.15.129:0/2648312310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 msgr2=0x556c6e23a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e23a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c6e23a7f0 con 0x556c6e1b6080 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.352+0000 7fed3bfff640 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed28009600 tx=0x7fed28012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4c906d300b91f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28011c80 con 0x556c6e1b6080 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fed28011e20 con 0x556c6e1b6080 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c6e1289d0 con 0x556c6e1b6080 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fed28005020 con 0x556c6e1b6080 2024-09-05T01:29:42.355 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed44fc2740 10 client.26305 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.353+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556c6e23bac0 con 0x556c6e1b6080 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556c6e1289d0 con 0x556c6e1b6080 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fed2801d3f0 con 0x556c6e1b6080 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 10 client.26305 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 10 client.26305 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 20 client.26305 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e23c2e0 0x556c6e23e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 4 client.26305 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.8, "await_for": 45.0}] 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.354+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.8, "await_for": 45.0}) -- 0x556c6e122d20 con 0x556c6e23c2e0 2024-09-05T01:29:42.356 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.355+0000 7fed40985640 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e23c2e0 0x556c6e23e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:42.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.355+0000 7fed1bfff640 10 client.26305 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.357 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.356+0000 7fed40985640 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e23c2e0 0x556c6e23e700 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb429e1b182d4842 server_cookie=e2ee3226fc882f2f in_seq=0 out_seq=0 2024-09-05T01:29:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.357+0000 7fe68eff5640 1 -- 172.21.15.129:0/2037935972 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d03e834a20 con 0x55d03e945450 2024-09-05T01:29:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.357+0000 7fe68eff5640 10 client.16857 handle_command_reply: tid=0 2024-09-05T01:29:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.357+0000 7fe6b4070740 1 client.16857 shutdown 2024-09-05T01:29:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.357+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 msgr2=0x55d03e8d2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.357+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8d2da0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe69c031d70 tx=0x7fe69c031740 comp rx=0 tx=0).stop 2024-09-05T01:29:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 shutdown_connections 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d03e945450 0x55d03e947870 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d03e8c9370 0x55d03e8d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d03e8c42e0 0x55d03e8d32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 --2- 172.21.15.129:0/2037935972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d03e8c39a0 0x55d03e8d2da0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.358+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 >> 172.21.15.129:0/2037935972 conn(0x55d03e8bf010 msgr2=0x55d03e8c0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.359+0000 7fe68eff5640 0 client.16857 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.359+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 shutdown_connections 2024-09-05T01:29:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.359+0000 7fe6b4070740 1 -- 172.21.15.129:0/2037935972 wait complete. 2024-09-05T01:29:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.359+0000 7fe6b4070740 20 client.16857 trim_cache size 0 max 16384 2024-09-05T01:29:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 msgr2=0x7fe6940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 0x7fe6940a5170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe6a401d000 tx=0x7fe6a401e000 comp rx=0 tx=0).stop 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 msgr2=0x7fe6ac080be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac080be0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe6a000e9f0 tx=0x7fe6a000eeb0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.363 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 shutdown_connections 2024-09-05T01:29:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe6940a2cb0 0x7fe6940a5170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe6ac109cd0 0x7fe6ac081c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe6ac109310 0x7fe6ac080be0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 --2- 172.21.15.129:0/771072474 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe6ac1084c0 0x7fe6ac0806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.364 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.362+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 >> 172.21.15.129:0/771072474 conn(0x7fe6ac06b6d0 msgr2=0x7fe6ac10c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.366 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.366+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 shutdown_connections 2024-09-05T01:29:42.367 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:42.366+0000 7fe6b12d7640 1 -- 172.21.15.129:0/771072474 wait complete. 2024-09-05T01:29:42.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:42 smithi129 ceph-mon[30979]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 802 op/s 2024-09-05T01:29:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:42 smithi194 ceph-mon[34941]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 802 op/s 2024-09-05T01:29:42.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:42 smithi183 ceph-mon[34589]: pgmap v785: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.4 MiB/s wr, 802 op/s 2024-09-05T01:29:42.665 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed1bfff640 1 -- 172.21.15.129:0/2648312310 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556c6e122d20 con 0x556c6e23c2e0 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed1bfff640 10 client.26305 handle_command_reply: tid=0 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 client.26305 shutdown 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 msgr2=0x556c6e1c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fed28009600 tx=0x7fed28012e70 comp rx=0 tx=0).stop 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 shutdown_connections 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556c6e23c2e0 0x556c6e23e700 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556c6e1c0940 0x556c6e23a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556c6e1b6080 0x556c6e1c4e70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 --2- 172.21.15.129:0/2648312310 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556c6e1b5760 0x556c6e1c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.666 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed1bfff640 0 client.26305 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 >> 172.21.15.129:0/2648312310 conn(0x556c6e1b1cf0 msgr2=0x556c6e1bc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 shutdown_connections 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.664+0000 7fed44fc2740 1 -- 172.21.15.129:0/2648312310 wait complete. 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed44fc2740 20 client.26305 trim_cache size 0 max 16384 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:29:42.667 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 54, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "8d6ec7e4": { 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 54, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 143.8, 2024-09-05T01:29:42.668 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:29:42.669 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:29:42.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 msgr2=0x7fed2c0a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 0x7fed2c0a51d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fed34005fd0 tx=0x7fed34004a40 comp rx=0 tx=0).stop 2024-09-05T01:29:42.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 msgr2=0x7fed3c1a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:42.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 0x7fed3c1a4640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fed3002a9f0 tx=0x7fed30003c00 comp rx=0 tx=0).stop 2024-09-05T01:29:42.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 shutdown_connections 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fed2c0a2d10 0x7fed2c0a51d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fed3c19e7c0 0x7fed3c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fed3c106a90 0x7fed3c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 --2- 172.21.15.129:0/4030683776 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fed3c100860 0x7fed3c1a4640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 >> 172.21.15.129:0/4030683776 conn(0x7fed3c077230 msgr2=0x7fed3c102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 shutdown_connections 2024-09-05T01:29:42.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:29:42.665+0000 7fed42be9640 1 -- 172.21.15.129:0/4030683776 wait complete. 2024-09-05T01:29:43.451 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8d6ec7e4', quiesce took 4.4 seconds. Will release after: 19.4 2024-09-05T01:29:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:44 smithi183 ceph-mon[34589]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-09-05T01:29:44.426 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:44 smithi129 ceph-mon[30979]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-09-05T01:29:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:44 smithi194 ceph-mon[34941]: pgmap v786: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.9 MiB/s wr, 779 op/s 2024-09-05T01:29:45.209 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 760.0 s, 357.4 tps, lat 123.100 ms stddev 53.680 2024-09-05T01:29:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:46 smithi183 ceph-mon[34589]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-09-05T01:29:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:46 smithi129 ceph-mon[30979]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-09-05T01:29:46.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:46 smithi194 ceph-mon[34941]: pgmap v787: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 MiB/s wr, 657 op/s 2024-09-05T01:29:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:48 smithi183 ceph-mon[34589]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-05T01:29:48.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:48 smithi129 ceph-mon[30979]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-05T01:29:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:48 smithi194 ceph-mon[34941]: pgmap v788: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.6 MiB/s wr, 620 op/s 2024-09-05T01:29:50.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 765.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:29:50.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:50 smithi129 ceph-mon[30979]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-05T01:29:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:50 smithi194 ceph-mon[34941]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-05T01:29:50.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:50 smithi183 ceph-mon[34589]: pgmap v789: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 3.6 MiB/s wr, 417 op/s 2024-09-05T01:29:52.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:52 smithi129 ceph-mon[30979]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-05T01:29:52.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:52 smithi194 ceph-mon[34941]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-05T01:29:52.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:52 smithi183 ceph-mon[34589]: pgmap v790: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.9 MiB/s wr, 334 op/s 2024-09-05T01:29:54.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:54 smithi129 ceph-mon[30979]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-05T01:29:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:54 smithi194 ceph-mon[34941]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-05T01:29:54.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:54 smithi183 ceph-mon[34589]: pgmap v791: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-09-05T01:29:55.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 770.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:29:55.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2789704871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 msgr2=0x7fbba8074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2789704871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba8074600 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009900 tx=0x7fbb9802a480 comp rx=0 tx=0).stop 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2789704871 shutdown_connections 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2789704871 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 0x7fbba810bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2789704871 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 0x7fbba8075030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2789704871 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba8074600 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2789704871 >> 172.21.15.129:0/2789704871 conn(0x7fbba80fd2e0 msgr2=0x7fbba80ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2789704871 shutdown_connections 2024-09-05T01:29:55.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.468+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2789704871 wait complete. 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.469+0000 7fbbae7d8640 1 Processor -- start 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.469+0000 7fbbae7d8640 1 -- start start 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 0x7fbba8199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 0x7fbba81a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810f8c0 con 0x7fbba80761b0 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810f740 con 0x7fbba8109a10 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbba810fa40 con 0x7fbba8074bd0 2024-09-05T01:29:55.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38612/0 (socket says 172.21.15.129:38612) 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 -- 172.21.15.129:0/2845320022 learned_addr learned my addr 172.21.15.129:0/2845320022 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba7fff640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 0x7fbba8199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 0x7fbba81a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 -- 172.21.15.129:0/2845320022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 msgr2=0x7fbba8199eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 0x7fbba8199eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 -- 172.21.15.129:0/2845320022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 msgr2=0x7fbba81a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 0x7fbba81a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 -- 172.21.15.129:0/2845320022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbba81a1ad0 con 0x7fbba80761b0 2024-09-05T01:29:55.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba77fe640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00d950 tx=0x7fbb9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba57fa640 1 -- 172.21.15.129:0/2845320022 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c017070 con 0x7fbba80761b0 2024-09-05T01:29:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba57fa640 1 -- 172.21.15.129:0/2845320022 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb9c009c30 con 0x7fbba80761b0 2024-09-05T01:29:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbba57fa640 1 -- 172.21.15.129:0/2845320022 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00f620 con 0x7fbba80761b0 2024-09-05T01:29:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.470+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbba81a1dc0 con 0x7fbba80761b0 2024-09-05T01:29:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.471+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbba81a2230 con 0x7fbba80761b0 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbba57fa640 1 -- 172.21.15.129:0/2845320022 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fbb9c0109a0 con 0x7fbba80761b0 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 Processor -- start 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 -- start start 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b772da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b773370 0x55e95b775830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e95b6d12f0 con 0x55e95b76d9a0 2024-09-05T01:29:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e95b679270 con 0x55e95b773370 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.473+0000 7fbbb1648740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e95b6cd2b0 con 0x55e95b76e2e0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38620/0 (socket says 172.21.15.129:38620) 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbba57fa640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 0x7fbb900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 -- 172.21.15.129:0/2553869084 learned_addr learned my addr 172.21.15.129:0/2553869084 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbba77fe640 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b773370 0x55e95b775830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbba7fff640 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b772da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 0x7fbb900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbba57fa640 1 -- 172.21.15.129:0/2845320022 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbb9c0cf000 con 0x7fbba80761b0 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 -- 172.21.15.129:0/2553869084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 msgr2=0x55e95b772da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b772da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 -- 172.21.15.129:0/2553869084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b773370 msgr2=0x55e95b775830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b773370 0x55e95b775830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 -- 172.21.15.129:0/2553869084 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95b678340 con 0x55e95b76d9a0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbba0012910 tx=0x7fbba0012d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c510331c40ef828 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbb8e7fc640 1 -- 172.21.15.129:0/2553869084 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0005020 con 0x55e95b76d9a0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbb8e7fc640 1 -- 172.21.15.129:0/2553869084 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba0005670 con 0x55e95b76d9a0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.474+0000 7fbb8e7fc640 1 -- 172.21.15.129:0/2553869084 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba001c6e0 con 0x55e95b76d9a0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 -- 172.21.15.129:0/2553869084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 msgr2=0x55e95b76dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbacd75640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 0x7fbb900a5170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbba001f000 tx=0x7fbba001e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbba0012910 tx=0x7fbba0012d10 comp rx=0 tx=0).stop 2024-09-05T01:29:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 -- 172.21.15.129:0/2553869084 shutdown_connections 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b773370 0x55e95b775830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b772da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 --2- 172.21.15.129:0/2553869084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b76d9a0 0x55e95b76dda0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 -- 172.21.15.129:0/2553869084 >> 172.21.15.129:0/2553869084 conn(0x55e95b769010 msgr2=0x55e95b769410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 -- 172.21.15.129:0/2553869084 shutdown_connections 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.475+0000 7fbbb1648740 1 -- 172.21.15.129:0/2553869084 wait complete. 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 -- start start 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 Processor -- start 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b777b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b7780d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e95b776340 con 0x55e95b773370 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e95b65a290 con 0x55e95b76d9a0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55e95b7764c0 con 0x55e95b76e2e0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbb1648740 10 client.0 _ll_register_callbacks cb 0x55e95b68b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38626/0 (socket says 172.21.15.129:38626) 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbbacd75640 1 -- 172.21.15.129:0/3802156573 learned_addr learned my addr 172.21.15.129:0/3802156573 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbba77fe640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b7780d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.476+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b777b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 -- 172.21.15.129:0/3802156573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 msgr2=0x55e95b7780d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b7780d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 -- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 msgr2=0x55e95b777b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b777b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 -- 172.21.15.129:0/3802156573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95b3e4210 con 0x55e95b773370 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbba77fe640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b7780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbba0005a50 tx=0x7fbba0005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f44e3f66aadf1e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba001cc10 con 0x55e95b773370 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbba002d070 con 0x55e95b773370 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba0005670 con 0x55e95b773370 2024-09-05T01:29:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbba003b050 con 0x55e95b773370 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbb1648740 10 client.16875 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e95b4b0620 con 0x55e95b773370 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba0005670 con 0x55e95b773370 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.477+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbba0003a90 con 0x55e95b773370 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 10 client.16875 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 10 client.16875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 20 client.16875 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7835e0 0x55e95b7859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 4 client.16875 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:29:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e95b6dea20 con 0x55e95b7835e0 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7835e0 0x55e95b7859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbb877fe640 10 client.16875 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.478+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7835e0 0x55e95b7859d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df54fd490c320e5 server_cookie=bafb1c6a8c238345 in_seq=0 out_seq=0 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbb877fe640 1 -- 172.21.15.129:0/3802156573 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e95b6dea20 con 0x55e95b7835e0 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbb877fe640 10 client.16875 handle_command_reply: tid=0 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 client.16875 shutdown 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 msgr2=0x55e95b77f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbba0005a50 tx=0x7fbba0005a80 comp rx=0 tx=0).stop 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 shutdown_connections 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7835e0 0x55e95b7859d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77f110 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b7780d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbb877fe640 0 client.16875 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3802156573 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b777b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 >> 172.21.15.129:0/3802156573 conn(0x55e95b769010 msgr2=0x55e95b76a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 shutdown_connections 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 1 -- 172.21.15.129:0/3802156573 wait complete. 2024-09-05T01:29:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.479+0000 7fbbb1648740 20 client.16875 trim_cache size 0 max 16384 2024-09-05T01:29:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.500+0000 7fbbb1648740 1 Processor -- start 2024-09-05T01:29:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.500+0000 7fbbb1648740 1 -- start start 2024-09-05T01:29:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b7693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81af1c0 con 0x55e95b773370 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81aca90 con 0x55e95b76d9a0 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbb1648740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81aa8e0 con 0x55e95b76e2e0 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38640/0 (socket says 172.21.15.129:38640) 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 -- 172.21.15.129:0/189818480 learned_addr learned my addr 172.21.15.129:0/189818480 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbba77fe640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbba7fff640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b7693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 -- 172.21.15.129:0/189818480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 msgr2=0x55e95b77f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 -- 172.21.15.129:0/189818480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 msgr2=0x55e95b7693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b7693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 -- 172.21.15.129:0/189818480 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e95b768e30 con 0x55e95b773370 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbba7fff640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b7693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbbacd75640 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010ee0 tx=0x7fbba0010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611927717075e581 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.501+0000 7fbb8dffb640 1 -- 172.21.15.129:0/189818480 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0010740 con 0x55e95b773370 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbb8dffb640 1 -- 172.21.15.129:0/189818480 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba00041d0 con 0x55e95b773370 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbb8dffb640 1 -- 172.21.15.129:0/189818480 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba00077a0 con 0x55e95b773370 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 -- 172.21.15.129:0/189818480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 msgr2=0x55e95b769930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010ee0 tx=0x7fbba0010f10 comp rx=0 tx=0).stop 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 -- 172.21.15.129:0/189818480 shutdown_connections 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b769930 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 --2- 172.21.15.129:0/189818480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b7693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 -- 172.21.15.129:0/189818480 >> 172.21.15.129:0/189818480 conn(0x55e95b769010 msgr2=0x55e95b777b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 -- 172.21.15.129:0/189818480 shutdown_connections 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.502+0000 7fbbb1648740 1 -- 172.21.15.129:0/189818480 wait complete. 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 -- start start 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 Processor -- start 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b77c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81aaf50 con 0x55e95b773370 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81ac3c0 con 0x55e95b76d9a0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fbba81ab620 con 0x55e95b76e2e0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbb1648740 10 client.0 _ll_register_callbacks cb 0x7fbb9c097770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbacd75640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:38644/0 (socket says 172.21.15.129:38644) 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbacd75640 1 -- 172.21.15.129:0/3579017423 learned_addr learned my addr 172.21.15.129:0/3579017423 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbba77fe640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b77c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.503+0000 7fbbacd75640 1 -- 172.21.15.129:0/3579017423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 msgr2=0x55e95b77c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbacd75640 1 -- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 msgr2=0x55e95b77c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b77c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbacd75640 1 -- 172.21.15.129:0/3579017423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e95b77dd00 con 0x55e95b773370 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbba77fe640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbacd75640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbba0012d50 tx=0x7fbba00126d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9294592b48856af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0007cd0 con 0x55e95b773370 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbba00041d0 con 0x55e95b773370 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fbba0021040 con 0x55e95b773370 2024-09-05T01:29:55.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbba0051050 con 0x55e95b773370 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 10 client.16881 fetch_fsmap learned FSMap version 34 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e95b77f010 con 0x55e95b773370 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbba00041d0 con 0x55e95b773370 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fbba003e070 con 0x55e95b773370 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 10 client.16881 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 10 client.16881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 20 client.16881 populate_metadata read hostname 'smithi129' 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7ee420 0x55e95b7f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 4 client.16881 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.504+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e95b6d8d70 con 0x55e95b7ee420 2024-09-05T01:29:55.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.505+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7ee420 0x55e95b7f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:29:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.505+0000 7fbba7fff640 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7ee420 0x55e95b7f0840 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2262cafcfe6ecab4 server_cookie=c63b7c793de741cd in_seq=0 out_seq=0 2024-09-05T01:29:55.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.505+0000 7fbb877fe640 10 client.16881 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbb877fe640 1 -- 172.21.15.129:0/3579017423 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e95b6d8d70 con 0x55e95b7ee420 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbb877fe640 10 client.16881 handle_command_reply: tid=0 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 client.16881 shutdown 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 msgr2=0x55e95b77d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbba0012d50 tx=0x7fbba00126d0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 shutdown_connections 2024-09-05T01:29:55.523 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55e95b7ee420 0x55e95b7f0840 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e95b773370 0x55e95b77d6f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55e95b76e2e0 0x55e95b77c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbbb1648740 1 --2- 172.21.15.129:0/3579017423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e95b76d9a0 0x55e95b77c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.521+0000 7fbb877fe640 0 client.16881 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 >> 172.21.15.129:0/3579017423 conn(0x55e95b769010 msgr2=0x55e95b7679a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 shutdown_connections 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbb1648740 1 -- 172.21.15.129:0/3579017423 wait complete. 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbb1648740 20 client.16881 trim_cache size 0 max 16384 2024-09-05T01:29:55.524 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:29:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:29:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:29:55.525 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 msgr2=0x7fbb900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 0x7fbb900a5170 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbba001f000 tx=0x7fbba001e000 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 msgr2=0x7fbba819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00d950 tx=0x7fbb9c00de10 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 shutdown_connections 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fbb900a2cb0 0x7fbb900a5170 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbba8109a10 0x7fbba81a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbba80761b0 0x7fbba819a3f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 --2- 172.21.15.129:0/2845320022 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fbba8074bd0 0x7fbba8199eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:29:55.526 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 >> 172.21.15.129:0/2845320022 conn(0x7fbba80fd2e0 msgr2=0x7fbba81059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:29:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 shutdown_connections 2024-09-05T01:29:55.527 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:29:55.522+0000 7fbbae7d8640 1 -- 172.21.15.129:0/2845320022 wait complete. 2024-09-05T01:29:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:56 smithi129 ceph-mon[30979]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 499 KiB/s wr, 66 op/s 2024-09-05T01:29:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:56 smithi194 ceph-mon[34941]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 499 KiB/s wr, 66 op/s 2024-09-05T01:29:56.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:56 smithi183 ceph-mon[34589]: pgmap v792: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 499 KiB/s wr, 66 op/s 2024-09-05T01:29:58.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:29:58 smithi129 ceph-mon[30979]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 317 KiB/s wr, 49 op/s 2024-09-05T01:29:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:29:58 smithi194 ceph-mon[34941]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 317 KiB/s wr, 49 op/s 2024-09-05T01:29:58.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:29:58 smithi183 ceph-mon[34589]: pgmap v793: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 317 KiB/s wr, 49 op/s 2024-09-05T01:30:00.210 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 775.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:30:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:00 smithi183 ceph-mon[34589]: overall HEALTH_OK 2024-09-05T01:30:00.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:00 smithi129 ceph-mon[30979]: overall HEALTH_OK 2024-09-05T01:30:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:00 smithi194 ceph-mon[34941]: overall HEALTH_OK 2024-09-05T01:30:01.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:01 smithi194 ceph-mon[34941]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 KiB/s wr, 36 op/s 2024-09-05T01:30:01.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:01 smithi183 ceph-mon[34589]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 KiB/s wr, 36 op/s 2024-09-05T01:30:01.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:01 smithi129 ceph-mon[30979]: pgmap v794: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 225 KiB/s wr, 36 op/s 2024-09-05T01:30:02.852 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:30:03.112 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:30:03.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:03 smithi129 ceph-mon[30979]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 212 KiB/s wr, 36 op/s 2024-09-05T01:30:03.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:03 smithi194 ceph-mon[34941]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 212 KiB/s wr, 36 op/s 2024-09-05T01:30:03.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:03 smithi183 ceph-mon[34589]: pgmap v795: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 212 KiB/s wr, 36 op/s 2024-09-05T01:30:03.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 -- 172.21.15.129:0/3926202662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24106f10 msgr2=0x7f9f2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:03.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 --2- 172.21.15.129:0/3926202662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24106f10 0x7f9f2410ea90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009ca0 tx=0x7f9f1c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 -- 172.21.15.129:0/3926202662 shutdown_connections 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 --2- 172.21.15.129:0/3926202662 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24106f10 0x7f9f2410ea90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 --2- 172.21.15.129:0/3926202662 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 0x7f9f241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 --2- 172.21.15.129:0/3926202662 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24101de0 0x7f9f241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 -- 172.21.15.129:0/3926202662 >> 172.21.15.129:0/3926202662 conn(0x7f9f240fd8c0 msgr2=0x7f9f240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 -- 172.21.15.129:0/3926202662 shutdown_connections 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.901+0000 7f9f28b95640 1 -- 172.21.15.129:0/3926202662 wait complete. 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.902+0000 7f9f28b95640 1 Processor -- start 2024-09-05T01:30:03.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.902+0000 7f9f28b95640 1 -- start start 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 0x7f9f24074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24106f10 0x7f9f24074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24111e90 con 0x7f9f24101de0 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24111d10 con 0x7f9f24102790 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24112010 con 0x7f9f24106f10 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43012/0 (socket says 172.21.15.129:43012) 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 -- 172.21.15.129:0/2912510259 learned_addr learned my addr 172.21.15.129:0/2912510259 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f19d9b640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 0x7f9f24074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:03.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f22d9d640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24106f10 0x7f9f24074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 -- 172.21.15.129:0/2912510259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24106f10 msgr2=0x7f9f24074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24106f10 0x7f9f24074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 -- 172.21.15.129:0/2912510259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 msgr2=0x7f9f24074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 0x7f9f24074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 -- 172.21.15.129:0/2912510259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f241a6840 con 0x7f9f24101de0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f2259c640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400b4a0 tx=0x7f9f1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14014070 con 0x7f9f24101de0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f14010ac0 con 0x7f9f24101de0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f241a6b30 con 0x7f9f24101de0 2024-09-05T01:30:03.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14017c30 con 0x7f9f24101de0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.903+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f241a7020 con 0x7f9f24101de0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.904+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f241021c0 con 0x7f9f24101de0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.906+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9f14017070 con 0x7f9f24101de0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.906+0000 7f9f1bfff640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 0x7f9f100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.906+0000 7f9f19d9b640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 0x7f9f100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.906+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9f140cec30 con 0x7f9f24101de0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.907+0000 7f9f19d9b640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 0x7f9f100a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9f241087d0 tx=0x7f9f0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:03.910+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f1409b780 con 0x7f9f24101de0 2024-09-05T01:30:04.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.119+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f24075a20 con 0x7f9f24101de0 2024-09-05T01:30:04.120 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.120+0000 7f9f1bfff640 1 -- 172.21.15.129:0/2912510259 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f9f24075a20 con 0x7f9f24101de0 2024-09-05T01:30:04.121 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:30:04.121 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:30:04.121 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 msgr2=0x7f9f100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 0x7f9f100a5040 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9f241087d0 tx=0x7f9f0c00a2d0 comp rx=0 tx=0).stop 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 msgr2=0x7f9f240744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400b4a0 tx=0x7f9f1400b960 comp rx=0 tx=0).stop 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 shutdown_connections 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9f100a2b80 0x7f9f100a5040 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:04.123 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9f24106f10 0x7f9f24074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:04.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f24102790 0x7f9f24074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:04.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 --2- 172.21.15.129:0/2912510259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f24101de0 0x7f9f240744f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:04.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 >> 172.21.15.129:0/2912510259 conn(0x7f9f240fd8c0 msgr2=0x7f9f240febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:04.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 shutdown_connections 2024-09-05T01:30:04.124 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:04.122+0000 7f9f28b95640 1 -- 172.21.15.129:0/2912510259 wait complete. 2024-09-05T01:30:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:04 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2912510259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:04 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2912510259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:04.562 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:04 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2912510259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:04.946 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 8d6ec7e4 --release --await' 2024-09-05T01:30:04.946 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 8d6ec7e4 --release --await 2024-09-05T01:30:05.200 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:30:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 780.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:30:05.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:05 smithi129 ceph-mon[30979]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:30:05.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:05 smithi194 ceph-mon[34941]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:30:05.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:05 smithi183 ceph-mon[34589]: pgmap v796: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:30:06.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 -- 172.21.15.129:0/520554828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 msgr2=0x7ffb5810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/520554828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5810ec40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009cd0 tx=0x7ffb5002a8e0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 -- 172.21.15.129:0/520554828 shutdown_connections 2024-09-05T01:30:06.193 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/520554828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5810ec40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/520554828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 0x7ffb58107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/520554828 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb58103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 -- 172.21.15.129:0/520554828 >> 172.21.15.129:0/520554828 conn(0x7ffb580fc7d0 msgr2=0x7ffb580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 -- 172.21.15.129:0/520554828 shutdown_connections 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 -- 172.21.15.129:0/520554828 wait complete. 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.089+0000 7ffb5e90b640 1 Processor -- start 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 -- start start 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 0x7ffb58198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.194 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb58111f50 con 0x7ffb581082a0 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb58111dd0 con 0x7ffb581036e0 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5e90b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581120d0 con 0x7ffb58102d30 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43022/0 (socket says 172.21.15.129:43022) 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 -- 172.21.15.129:0/2085830967 learned_addr learned my addr 172.21.15.129:0/2085830967 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb57fff640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 0x7ffb58198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.195 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 -- 172.21.15.129:0/2085830967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 msgr2=0x7ffb581983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb581983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 -- 172.21.15.129:0/2085830967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 msgr2=0x7ffb58198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 0x7ffb58198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 -- 172.21.15.129:0/2085830967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5819d380 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb57fff640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.090+0000 7ffb5cea8640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffb50005cc0 tx=0x7ffb50003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.091+0000 7ffb55ffb640 1 -- 172.21.15.129:0/2085830967 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb50004240 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.091+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5819d610 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.091+0000 7ffb55ffb640 1 -- 172.21.15.129:0/2085830967 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb500305a0 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.091+0000 7ffb55ffb640 1 -- 172.21.15.129:0/2085830967 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb500466c0 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.091+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb581a4f50 con 0x7ffb581082a0 2024-09-05T01:30:06.196 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.093+0000 7ffb55ffb640 1 -- 172.21.15.129:0/2085830967 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ffb50030ac0 con 0x7ffb581082a0 2024-09-05T01:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.093+0000 7ffb60ce4740 1 Processor -- start 2024-09-05T01:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.093+0000 7ffb60ce4740 1 -- start start 2024-09-05T01:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b5997d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.202 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d4b58f2620 con 0x55d4b598d060 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4b5899e50 con 0x55d4b598d980 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d4b58ee5e0 con 0x55d4b5998240 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43030/0 (socket says 172.21.15.129:43030) 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 -- 172.21.15.129:0/390401945 learned_addr learned my addr 172.21.15.129:0/390401945 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb55ffb640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 0x7ffb440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.203 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb57fff640 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b5997d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 -- 172.21.15.129:0/390401945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 msgr2=0x55d4b599a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 -- 172.21.15.129:0/390401945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 msgr2=0x55d4b5997d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b5997d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 -- 172.21.15.129:0/390401945 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b588ad40 con 0x55d4b598d060 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb5cea8640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 0x7ffb440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb55ffb640 1 -- 172.21.15.129:0/2085830967 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb5002e030 con 0x7ffb581082a0 2024-09-05T01:30:06.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005940 tx=0x7ffb40010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b839e93fe31bf6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4e7fc640 1 -- 172.21.15.129:0/390401945 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40016030 con 0x55d4b598d060 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4e7fc640 1 -- 172.21.15.129:0/390401945 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40010d50 con 0x55d4b598d060 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.094+0000 7ffb4e7fc640 1 -- 172.21.15.129:0/390401945 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40012480 con 0x55d4b598d060 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- 172.21.15.129:0/390401945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 msgr2=0x55d4b598d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb5cea8640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 0x7ffb440a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb500c5670 tx=0x7ffb50087b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005940 tx=0x7ffb40010370 comp rx=0 tx=0).stop 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- 172.21.15.129:0/390401945 shutdown_connections 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b5997d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/390401945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b598d440 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- 172.21.15.129:0/390401945 >> 172.21.15.129:0/390401945 conn(0x55d4b59895f0 msgr2=0x55d4b59899d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- 172.21.15.129:0/390401945 shutdown_connections 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- 172.21.15.129:0/390401945 wait complete. 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.095+0000 7ffb60ce4740 1 -- start start 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 Processor -- start 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b5995180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b5995c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d4b599ae10 con 0x55d4b598d980 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d4b587af10 con 0x55d4b598d060 2024-09-05T01:30:06.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55d4b599af90 con 0x55d4b5998240 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb60ce4740 10 client.0 _ll_register_callbacks cb 0x55d4b589edb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43042/0 (socket says 172.21.15.129:43042) 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 -- 172.21.15.129:0/4088080244 learned_addr learned my addr 172.21.15.129:0/4088080244 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b5995180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 -- 172.21.15.129:0/4088080244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 msgr2=0x55d4b5995c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b5995c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 -- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 msgr2=0x55d4b5995180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b5995180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.096+0000 7ffb57fff640 1 -- 172.21.15.129:0/4088080244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b5601580 con 0x55d4b598d980 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb48010870 tx=0x7ffb48010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a03d393fa88425 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800ed40 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb48011440 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb40010d50 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffb48011640 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 10 client.16899 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d4b56cb960 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb40010d50 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ffb4801c7f0 con 0x55d4b598d980 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 10 client.16899 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 10 client.16899 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:30:06.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 20 client.16899 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b59a7eb0 0x55d4b59aa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 4 client.16899 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.097+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4b58ffb90 con 0x55d4b59a7eb0 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.098+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b59a7eb0 0x55d4b59aa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.098+0000 7ffb377fe640 10 client.16899 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.098+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b59a7eb0 0x55d4b59aa350 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca03524c183b7e6 server_cookie=93ec40e527825478 in_seq=0 out_seq=0 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb377fe640 1 -- 172.21.15.129:0/4088080244 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4b58ffb90 con 0x55d4b59a7eb0 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb377fe640 10 client.16899 handle_command_reply: tid=0 2024-09-05T01:30:06.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 client.16899 shutdown 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 msgr2=0x55d4b59956c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb48010870 tx=0x7ffb48010d30 comp rx=0 tx=0).stop 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 shutdown_connections 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b59a7eb0 0x55d4b59aa350 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b5995c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b59956c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb377fe640 0 client.16899 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:06.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4088080244 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b5995180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 >> 172.21.15.129:0/4088080244 conn(0x55d4b59895f0 msgr2=0x55d4b598f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 shutdown_connections 2024-09-05T01:30:06.211 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4088080244 wait complete. 2024-09-05T01:30:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.099+0000 7ffb60ce4740 20 client.16899 trim_cache size 0 max 16384 2024-09-05T01:30:06.212 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 Processor -- start 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 -- start start 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b598fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b59afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581b1550 con 0x55d4b598d980 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581aee20 con 0x55d4b598d060 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581acc70 con 0x55d4b5998240 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43052/0 (socket says 172.21.15.129:43052) 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 -- 172.21.15.129:0/4089557829 learned_addr learned my addr 172.21.15.129:0/4089557829 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:06.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 -- 172.21.15.129:0/4089557829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 msgr2=0x55d4b59afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b59afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 -- 172.21.15.129:0/4089557829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 msgr2=0x55d4b598fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b598fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.116+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b598fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb57fff640 1 -- 172.21.15.129:0/4089557829 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4b59aa9d0 con 0x55d4b598d980 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b598fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb4801ce00 tx=0x7ffb480114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2484d0728511d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb4dffb640 1 -- 172.21.15.129:0/4089557829 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4802d020 con 0x55d4b598d980 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb4dffb640 1 -- 172.21.15.129:0/4089557829 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4801c480 con 0x55d4b598d980 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.117+0000 7ffb4dffb640 1 -- 172.21.15.129:0/4089557829 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb48005470 con 0x55d4b598d980 2024-09-05T01:30:06.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4089557829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 msgr2=0x55d4b5988640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb4801ce00 tx=0x7ffb480114c0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4089557829 shutdown_connections 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b59afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d980 0x55d4b5988640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4089557829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d060 0x55d4b598fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4089557829 >> 172.21.15.129:0/4089557829 conn(0x55d4b59895f0 msgr2=0x55d4b5987f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4089557829 shutdown_connections 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4089557829 wait complete. 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- start start 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 Processor -- start 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.215 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b599bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581ad2e0 con 0x55d4b598d060 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581ae750 con 0x55d4b598d980 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.118+0000 7ffb60ce4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ffb581ad9b0 con 0x55d4b5998240 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb60ce4740 10 client.0 _ll_register_callbacks cb 0x7ffb480103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43068/0 (socket says 172.21.15.129:43068) 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 -- 172.21.15.129:0/4282821808 learned_addr learned my addr 172.21.15.129:0/4282821808 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b599bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 -- 172.21.15.129:0/4282821808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 msgr2=0x55d4b599c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 -- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 msgr2=0x55d4b599bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b599bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 -- 172.21.15.129:0/4282821808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4b599cbe0 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb4ffff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005910 tx=0x7ffb4000c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ac737a2af2737e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb377fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40016040 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ffb40004220 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4801c480 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffb40004420 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb60ce4740 10 client.16905 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.119+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d4b5a12d50 con 0x55d4b598d060 2024-09-05T01:30:06.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4801c480 con 0x55d4b598d060 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ffb40004ec0 con 0x55d4b598d060 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 10 client.16905 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 10 client.16905 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 20 client.16905 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b5a132e0 0x55d4b5a156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 4 client.16905 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8d6ec7e4", "release": true, "await": true}] 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8d6ec7e4", "release": true, "await": true}) -- 0x55d4b58fa0a0 con 0x55d4b5a132e0 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b5a132e0 0x55d4b5a156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb377fe640 10 client.16905 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:06.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.120+0000 7ffb57fff640 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b5a132e0 0x55d4b5a156d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a520558f47239ee7 server_cookie=a4d82f43c6afea4a in_seq=0 out_seq=0 2024-09-05T01:30:06.219 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:30:06.219 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:30:06.219 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:30:06.219 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 56, 2024-09-05T01:30:06.219 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "8d6ec7e4": { 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 56, 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:30:06.220 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 143.8, 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:06.221 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb377fe640 1 -- 172.21.15.129:0/4282821808 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d4b58fa0a0 con 0x55d4b5a132e0 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb377fe640 10 client.16905 handle_command_reply: tid=0 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 client.16905 shutdown 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 msgr2=0x55d4b59a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.222 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ffb40005910 tx=0x7ffb4000c350 comp rx=0 tx=0).stop 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 shutdown_connections 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55d4b5a132e0 0x55d4b5a156d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb377fe640 0 client.16905 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55d4b5998240 0x55d4b599c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d4b598d980 0x55d4b599bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 --2- 172.21.15.129:0/4282821808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4b598d060 0x55d4b59a0a40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 >> 172.21.15.129:0/4282821808 conn(0x55d4b59895f0 msgr2=0x55d4b5997200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 shutdown_connections 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.177+0000 7ffb60ce4740 1 -- 172.21.15.129:0/4282821808 wait complete. 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb60ce4740 20 client.16905 trim_cache size 0 max 16384 2024-09-05T01:30:06.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 msgr2=0x7ffb440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 0x7ffb440a5040 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb500c5670 tx=0x7ffb50087b00 comp rx=0 tx=0).stop 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 msgr2=0x7ffb5819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffb50005cc0 tx=0x7ffb50003ba0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 shutdown_connections 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ffb440a2b80 0x7ffb440a5040 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb581082a0 0x7ffb5819cca0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffb581036e0 0x7ffb58198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 --2- 172.21.15.129:0/2085830967 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ffb58102d30 0x7ffb581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:06.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.178+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 >> 172.21.15.129:0/2085830967 conn(0x7ffb580fc7d0 msgr2=0x7ffb58105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:06.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.179+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 shutdown_connections 2024-09-05T01:30:06.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:06.179+0000 7ffb5e90b640 1 -- 172.21.15.129:0/2085830967 wait complete. 2024-09-05T01:30:06.978 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8d6ec7e4', total seconds elapsed: 27.9 2024-09-05T01:30:07.243 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:07 smithi129 ceph-mon[30979]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T01:30:07.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:07 smithi194 ceph-mon[34941]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T01:30:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:07 smithi183 ceph-mon[34589]: pgmap v797: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T01:30:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 -- 172.21.15.129:0/500022690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 msgr2=0x7ff084101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 --2- 172.21.15.129:0/500022690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff084101650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff078009900 tx=0x7ff07802c650 comp rx=0 tx=0).stop 2024-09-05T01:30:08.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 -- 172.21.15.129:0/500022690 shutdown_connections 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 --2- 172.21.15.129:0/500022690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 0x7ff084108420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 --2- 172.21.15.129:0/500022690 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff084101650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 --2- 172.21.15.129:0/500022690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff0841003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.658+0000 7ff089a6e640 1 -- 172.21.15.129:0/500022690 >> 172.21.15.129:0/500022690 conn(0x7ff0840fb740 msgr2=0x7ff0840fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.659+0000 7ff089a6e640 1 -- 172.21.15.129:0/500022690 shutdown_connections 2024-09-05T01:30:08.661 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.659+0000 7ff089a6e640 1 -- 172.21.15.129:0/500022690 wait complete. 2024-09-05T01:30:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.660+0000 7ff089a6e640 1 Processor -- start 2024-09-05T01:30:08.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.660+0000 7ff089a6e640 1 -- start start 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff08419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 0x7ff0841a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43084/0 (socket says 172.21.15.129:43084) 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff088a6c640 1 -- 172.21.15.129:0/2204761115 learned_addr learned my addr 172.21.15.129:0/2204761115 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410a990 con 0x7ff0840fffe0 2024-09-05T01:30:08.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410a810 con 0x7ff084101b90 2024-09-05T01:30:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff08410ab10 con 0x7ff0841011d0 2024-09-05T01:30:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff08926d640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 0x7ff0841a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.661+0000 7ff083fff640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff08419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 -- 172.21.15.129:0/2204761115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 msgr2=0x7ff08419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff08419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 -- 172.21.15.129:0/2204761115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 msgr2=0x7ff0841a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 0x7ff0841a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0841a5eb0 con 0x7ff0840fffe0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b470 tx=0x7ff07000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff081ffb640 1 -- 172.21.15.129:0/2204761115 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070015020 con 0x7ff0840fffe0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff081ffb640 1 -- 172.21.15.129:0/2204761115 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff070011d40 con 0x7ff0840fffe0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff081ffb640 1 -- 172.21.15.129:0/2204761115 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07000f340 con 0x7ff0840fffe0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.662+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0841a61a0 con 0x7ff0840fffe0 2024-09-05T01:30:08.665 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.663+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a6670 con 0x7ff0840fffe0 2024-09-05T01:30:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.667+0000 7ff08da6b740 1 Processor -- start 2024-09-05T01:30:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.667+0000 7ff081ffb640 1 -- 172.21.15.129:0/2204761115 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff070018020 con 0x7ff0840fffe0 2024-09-05T01:30:08.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.667+0000 7ff08da6b740 1 -- start start 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc653df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc6545a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ecc64a12f0 con 0x55ecc6543540 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6449270 con 0x55ecc653e4b0 2024-09-05T01:30:08.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08da6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ecc649d2b0 con 0x55ecc653db70 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53524/0 (socket says 172.21.15.129:53524) 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff088a6c640 1 -- 172.21.15.129:0/1681043557 learned_addr learned my addr 172.21.15.129:0/1681043557 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff08926d640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc653df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.668+0000 7ff083fff640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc6545a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 -- 172.21.15.129:0/1681043557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 msgr2=0x55ecc653df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc653df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 -- 172.21.15.129:0/1681043557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 msgr2=0x55ecc6545a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc6545a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 -- 172.21.15.129:0/1681043557 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6448340 con 0x55ecc653e4b0 2024-09-05T01:30:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff083fff640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc6545a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:30:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0700d2eb0 tx=0x7ff0700d57d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d8e084ed97579e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff081ffb640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 0x7ff0740a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.669+0000 7ff06effd640 1 -- 172.21.15.129:0/1681043557 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700e4070 con 0x55ecc653e4b0 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.670+0000 7ff06effd640 1 -- 172.21.15.129:0/1681043557 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0700700d0 con 0x55ecc653e4b0 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.670+0000 7ff06effd640 1 -- 172.21.15.129:0/1681043557 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07009abd0 con 0x55ecc653e4b0 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.670+0000 7ff08926d640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 0x7ff0740a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.670+0000 7ff081ffb640 1 -- 172.21.15.129:0/2204761115 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff0700cee80 con 0x7ff0840fffe0 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- 172.21.15.129:0/1681043557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 msgr2=0x55ecc6542f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff0700d2eb0 tx=0x7ff0700d57d0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- 172.21.15.129:0/1681043557 shutdown_connections 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc6545a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6542f70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1681043557 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc653df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- 172.21.15.129:0/1681043557 >> 172.21.15.129:0/1681043557 conn(0x55ecc65391e0 msgr2=0x55ecc65395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08926d640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 0x7ff0740a5140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005fd0 tx=0x7ff07c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- 172.21.15.129:0/1681043557 shutdown_connections 2024-09-05T01:30:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- 172.21.15.129:0/1681043557 wait complete. 2024-09-05T01:30:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.671+0000 7ff08da6b740 1 -- start start 2024-09-05T01:30:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 Processor -- start 2024-09-05T01:30:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6548360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6546510 con 0x55ecc6543540 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ecc642a290 con 0x55ecc653e4b0 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6546690 con 0x55ecc653db70 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.672+0000 7ff08da6b740 10 client.0 _ll_register_callbacks cb 0x55ecc645b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:43218/0 (socket says 172.21.15.129:43218) 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 -- 172.21.15.129:0/2087304320 learned_addr learned my addr 172.21.15.129:0/2087304320 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff08926d640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff083fff640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6548360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 -- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 msgr2=0x55ecc6548360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6548360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 -- 172.21.15.129:0/2087304320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 msgr2=0x55ecc654f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 -- 172.21.15.129:0/2087304320 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc61b4210 con 0x55ecc653db70 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff088a6c640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0700108f0 tx=0x7ff0700cf0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9fdf5e5b8409275 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff063fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.673+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0700782a0 con 0x55ecc653db70 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.674+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0700700d0 con 0x55ecc653db70 2024-09-05T01:30:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.674+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff070099570 con 0x55ecc653db70 2024-09-05T01:30:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.674+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff07009abd0 con 0x55ecc653db70 2024-09-05T01:30:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.674+0000 7ff08da6b740 10 client.26341 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.674+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ecc6280620 con 0x55ecc653db70 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.675+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff07009b970 con 0x55ecc653db70 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.675+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff0700b40d0 con 0x55ecc653db70 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.675+0000 7ff08da6b740 10 client.26341 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.675+0000 7ff08da6b740 10 client.26341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.675+0000 7ff08da6b740 20 client.26341 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.676+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65536a0 0x55ecc6555a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.676+0000 7ff08da6b740 4 client.26341 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.676+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0700700d0 con 0x55ecc65536a0 2024-09-05T01:30:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.680+0000 7ff083fff640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65536a0 0x55ecc6555a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.682+0000 7ff083fff640 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65536a0 0x55ecc6555a90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c1873510217ef4 server_cookie=a2167eff5d7fe629 in_seq=0 out_seq=0 2024-09-05T01:30:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.682+0000 7ff063fff640 10 client.26341 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.683+0000 7ff063fff640 1 -- 172.21.15.129:0/2087304320 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff0700700d0 con 0x55ecc65536a0 2024-09-05T01:30:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.683+0000 7ff063fff640 10 client.26341 handle_command_reply: tid=0 2024-09-05T01:30:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.683+0000 7ff08da6b740 1 client.26341 shutdown 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 msgr2=0x55ecc6547e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0700108f0 tx=0x7ff0700cf0d0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 shutdown_connections 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65536a0 0x55ecc6555a90 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff063fff640 0 client.26341 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc6548360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 --2- 172.21.15.129:0/2087304320 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc653db70 0x55ecc6547e20 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0700108f0 tx=0x7ff0700cf0d0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.684+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 >> 172.21.15.129:0/2087304320 conn(0x55ecc65391e0 msgr2=0x55ecc653a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.685+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 shutdown_connections 2024-09-05T01:30:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.686+0000 7ff08da6b740 1 -- 172.21.15.129:0/2087304320 wait complete. 2024-09-05T01:30:08.687 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.686+0000 7ff08da6b740 20 client.26341 trim_cache size 0 max 16384 2024-09-05T01:30:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 Processor -- start 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 -- start start 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc65395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc6539b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841b36d0 con 0x55ecc6543540 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841b0fa0 con 0x55ecc653e4b0 2024-09-05T01:30:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.717+0000 7ff08da6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841aedf0 con 0x55ecc65536a0 2024-09-05T01:30:08.721 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.720+0000 7ff083fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.720+0000 7ff083fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43110/0 (socket says 172.21.15.129:43110) 2024-09-05T01:30:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.720+0000 7ff083fff640 1 -- 172.21.15.129:0/1929739353 learned_addr learned my addr 172.21.15.129:0/1929739353 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.720+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc65395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.720+0000 7ff08926d640 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc6539b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.722 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 -- 172.21.15.129:0/1929739353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 msgr2=0x55ecc6539b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc6539b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 -- 172.21.15.129:0/1929739353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 msgr2=0x55ecc654f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 -- 172.21.15.129:0/1929739353 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6539000 con 0x55ecc653e4b0 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff088a6c640 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc65395c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0700d6000 tx=0x7ff0700d2840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9606e66565e6ef46 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff06e7fc640 1 -- 172.21.15.129:0/1929739353 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07009b040 con 0x55ecc653e4b0 2024-09-05T01:30:08.723 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.721+0000 7ff06e7fc640 1 -- 172.21.15.129:0/1929739353 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0700d2610 con 0x55ecc653e4b0 2024-09-05T01:30:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.722+0000 7ff06e7fc640 1 -- 172.21.15.129:0/1929739353 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070097510 con 0x55ecc653e4b0 2024-09-05T01:30:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- 172.21.15.129:0/1929739353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 msgr2=0x55ecc65395c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.724 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc65395c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff0700d6000 tx=0x7ff0700d2840 comp rx=0 tx=0).stop 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- 172.21.15.129:0/1929739353 shutdown_connections 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc6539b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 --2- 172.21.15.129:0/1929739353 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc65395c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- 172.21.15.129:0/1929739353 >> 172.21.15.129:0/1929739353 conn(0x55ecc65391e0 msgr2=0x55ecc6533ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.725 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- 172.21.15.129:0/1929739353 shutdown_connections 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- 172.21.15.129:0/1929739353 wait complete. 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 -- start start 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.723+0000 7ff08da6b740 1 Processor -- start 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc654df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.726 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841af460 con 0x55ecc6543540 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841b08d0 con 0x55ecc653e4b0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff0841afb30 con 0x55ecc65536a0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08da6b740 10 client.0 _ll_register_callbacks cb 0x7ff0700e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff083fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff088a6c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff08926d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc654df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff083fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:43112/0 (socket says 172.21.15.129:43112) 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff083fff640 1 -- 172.21.15.129:0/519519525 learned_addr learned my addr 172.21.15.129:0/519519525 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.724+0000 7ff088a6c640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53546/0 (socket says 172.21.15.129:53546) 2024-09-05T01:30:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 -- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 msgr2=0x55ecc654ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 -- 172.21.15.129:0/519519525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 msgr2=0x55ecc654cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 -- 172.21.15.129:0/519519525 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc654e560 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff088a6c640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08926d640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc654df50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00abf0 tx=0x7ff07c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d7652ac908f9f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff063fff640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02d060 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff07c028440 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.725+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0700d2610 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff07c028640 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 10 client.26347 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ecc65cf070 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0700d2610 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7ff07c00b040 con 0x55ecc65536a0 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 10 client.26347 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 10 client.26347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 20 client.26347 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65cf9d0 0x55ecc65d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 4 client.26347 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.726+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ecc64aea20 con 0x55ecc65cf9d0 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.727+0000 7ff088a6c640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65cf9d0 0x55ecc65d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.727+0000 7ff088a6c640 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65cf9d0 0x55ecc65d1df0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e8f72969d51d7b server_cookie=a80623d3b61e4f2d in_seq=0 out_seq=0 2024-09-05T01:30:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.727+0000 7ff063fff640 10 client.26347 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.781+0000 7ff063fff640 1 -- 172.21.15.129:0/519519525 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ecc64aea20 con 0x55ecc65cf9d0 2024-09-05T01:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.781+0000 7ff063fff640 10 client.26347 handle_command_reply: tid=0 2024-09-05T01:30:08.783 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.782+0000 7ff08da6b740 1 client.26347 shutdown 2024-09-05T01:30:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 msgr2=0x55ecc654df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc654df50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff07c00abf0 tx=0x7ff07c004ab0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.785 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 shutdown_connections 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55ecc65cf9d0 0x55ecc65d1df0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55ecc65536a0 0x55ecc654df50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ecc6543540 0x55ecc654cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 --2- 172.21.15.129:0/519519525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ecc653e4b0 0x55ecc654ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 >> 172.21.15.129:0/519519525 conn(0x55ecc65391e0 msgr2=0x55ecc6537a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff063fff640 0 client.26347 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.783+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 shutdown_connections 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.784+0000 7ff08da6b740 1 -- 172.21.15.129:0/519519525 wait complete. 2024-09-05T01:30:08.786 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.784+0000 7ff08da6b740 20 client.26347 trim_cache size 0 max 16384 2024-09-05T01:30:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:30:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:30:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:30:08.787 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 msgr2=0x7ff0740a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 0x7ff0740a5140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff07c005fd0 tx=0x7ff07c010040 comp rx=0 tx=0).stop 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 msgr2=0x7ff08419e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff07000b470 tx=0x7ff07000b930 comp rx=0 tx=0).stop 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 shutdown_connections 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff0740a2c80 0x7ff0740a5140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff084101b90 0x7ff0841a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff0841011d0 0x7ff08419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 --2- 172.21.15.129:0/2204761115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0840fffe0 0x7ff08419e290 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 >> 172.21.15.129:0/2204761115 conn(0x7ff0840fb740 msgr2=0x7ff0841023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 shutdown_connections 2024-09-05T01:30:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:08.785+0000 7ff089a6e640 1 -- 172.21.15.129:0/2204761115 wait complete. 2024-09-05T01:30:09.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:09 smithi129 ceph-mon[30979]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-05T01:30:09.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:09 smithi194 ceph-mon[34941]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-05T01:30:09.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:09 smithi183 ceph-mon[34589]: pgmap v798: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 559 KiB/s wr, 75 op/s 2024-09-05T01:30:10.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 785.0 s, 672.1 tps, lat 826.289 ms stddev 4043.496 2024-09-05T01:30:11.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:11 smithi129 ceph-mon[30979]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-05T01:30:11.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:11 smithi194 ceph-mon[34941]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-05T01:30:11.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:11 smithi183 ceph-mon[34589]: pgmap v799: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-05T01:30:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:13 smithi129 ceph-mon[30979]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-05T01:30:13.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:13 smithi194 ceph-mon[34941]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-05T01:30:13.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:13 smithi183 ceph-mon[34589]: pgmap v800: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 2.0 MiB/s wr, 236 op/s 2024-09-05T01:30:15.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 790.0 s, 752.9 tps, lat 132.790 ms stddev 129.164 2024-09-05T01:30:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:15 smithi129 ceph-mon[30979]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 372 op/s 2024-09-05T01:30:15.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:15 smithi194 ceph-mon[34941]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 372 op/s 2024-09-05T01:30:15.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:15 smithi183 ceph-mon[34589]: pgmap v801: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 3.1 MiB/s wr, 372 op/s 2024-09-05T01:30:17.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:17 smithi129 ceph-mon[30979]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-09-05T01:30:17.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:17 smithi194 ceph-mon[34941]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-09-05T01:30:17.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:17 smithi183 ceph-mon[34589]: pgmap v802: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.7 MiB/s wr, 430 op/s 2024-09-05T01:30:19.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:19 smithi129 ceph-mon[30979]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 632 op/s 2024-09-05T01:30:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:19 smithi194 ceph-mon[34941]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 632 op/s 2024-09-05T01:30:19.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:19 smithi183 ceph-mon[34589]: pgmap v803: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.6 MiB/s wr, 632 op/s 2024-09-05T01:30:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 795.0 s, 836.2 tps, lat 121.467 ms stddev 98.219 2024-09-05T01:30:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:21 smithi129 ceph-mon[30979]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-09-05T01:30:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:21 smithi194 ceph-mon[34941]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-09-05T01:30:21.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:21 smithi183 ceph-mon[34589]: pgmap v804: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.0 MiB/s wr, 684 op/s 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 -- 172.21.15.129:0/1027370100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 msgr2=0x7f7bf01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/1027370100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 0x7f7bf01020b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009900 tx=0x7f7bd802a410 comp rx=0 tx=0).stop 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 -- 172.21.15.129:0/1027370100 shutdown_connections 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/1027370100 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 0x7f7bf0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/1027370100 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 0x7f7bf01020b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/1027370100 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf0100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 -- 172.21.15.129:0/1027370100 >> 172.21.15.129:0/1027370100 conn(0x7f7bf00fc1e0 msgr2=0x7f7bf00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 -- 172.21.15.129:0/1027370100 shutdown_connections 2024-09-05T01:30:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.922+0000 7f7bf52e6640 1 -- 172.21.15.129:0/1027370100 wait complete. 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 Processor -- start 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 -- start start 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 0x7f7bf019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 0x7f7bf01a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010c570 con 0x7f7bf0101c30 2024-09-05T01:30:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010c3f0 con 0x7f7bf01025f0 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.923+0000 7f7bf52e6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf010c6f0 con 0x7f7bf0100a40 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:41382/0 (socket says 172.21.15.129:41382) 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 -- 172.21.15.129:0/337643571 learned_addr learned my addr 172.21.15.129:0/337643571 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 0x7f7bf01a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 -- 172.21.15.129:0/337643571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 msgr2=0x7f7bf01a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 0x7f7bf01a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 -- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 msgr2=0x7f7bf019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 0x7f7bf019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 -- 172.21.15.129:0/337643571 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf01a5f80 con 0x7f7bf0100a40 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.924+0000 7f7beeffd640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d950 tx=0x7f7be000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.925+0000 7f7be7fff640 1 -- 172.21.15.129:0/337643571 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0017070 con 0x7f7bf0100a40 2024-09-05T01:30:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.925+0000 7f7be7fff640 1 -- 172.21.15.129:0/337643571 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be0009c30 con 0x7f7bf0100a40 2024-09-05T01:30:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.925+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf01a6270 con 0x7f7bf0100a40 2024-09-05T01:30:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.925+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a67c0 con 0x7f7bf0100a40 2024-09-05T01:30:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.925+0000 7f7be7fff640 1 -- 172.21.15.129:0/337643571 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be000f620 con 0x7f7bf0100a40 2024-09-05T01:30:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.928+0000 7f7be7fff640 1 -- 172.21.15.129:0/337643571 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7be001a070 con 0x7f7bf0100a40 2024-09-05T01:30:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.928+0000 7f7bf8080740 1 Processor -- start 2024-09-05T01:30:21.929 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.928+0000 7f7bf8080740 1 -- start start 2024-09-05T01:30:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c0fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c12830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ad1b6e2f0 con 0x556ad1c0b2e0 2024-09-05T01:30:21.930 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ad1b16270 con 0x556ad1c0a9a0 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.929+0000 7f7bf8080740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ad1b6a2b0 con 0x556ad1c10370 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7beeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c0fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53198/0 (socket says 172.21.15.129:53198) 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7beeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c0fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47866/0 (socket says 172.21.15.129:47866) 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 -- 172.21.15.129:0/461114094 learned_addr learned my addr 172.21.15.129:0/461114094 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:21.931 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c12830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 -- 172.21.15.129:0/461114094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 msgr2=0x556ad1c12830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c12830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 -- 172.21.15.129:0/461114094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 msgr2=0x556ad1c0fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c0fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.930+0000 7f7bef7fe640 1 -- 172.21.15.129:0/461114094 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad1b15340 con 0x556ad1c0a9a0 2024-09-05T01:30:21.932 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7be7fff640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 0x7f7bdc0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009fd0 tx=0x7f7be80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e041e0ea89478e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7beeffd640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 0x7f7bdc0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7be57fa640 1 -- 172.21.15.129:0/461114094 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004d20 con 0x556ad1c0a9a0 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7be57fa640 1 -- 172.21.15.129:0/461114094 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8004050 con 0x556ad1c0a9a0 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7be7fff640 1 -- 172.21.15.129:0/337643571 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7be00ced80 con 0x7f7bf0100a40 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.931+0000 7f7be57fa640 1 -- 172.21.15.129:0/461114094 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8005280 con 0x556ad1c0a9a0 2024-09-05T01:30:21.933 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.932+0000 7f7bf8080740 1 -- 172.21.15.129:0/461114094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 msgr2=0x556ad1c0ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.932+0000 7f7bf8080740 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009fd0 tx=0x7f7be80118d0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.934 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.933+0000 7f7beeffd640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 0x7f7bdc0a5260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7be00cc7e0 tx=0x7f7be00d2860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.935+0000 7f7bf8080740 1 -- 172.21.15.129:0/461114094 shutdown_connections 2024-09-05T01:30:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.935+0000 7f7bf8080740 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c12830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.935+0000 7f7bf8080740 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c0fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.935+0000 7f7bf8080740 1 --2- 172.21.15.129:0/461114094 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c0ada0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.936 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.935+0000 7f7bf8080740 1 -- 172.21.15.129:0/461114094 >> 172.21.15.129:0/461114094 conn(0x556ad1c06010 msgr2=0x556ad1c06410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.939+0000 7f7bf8080740 1 -- 172.21.15.129:0/461114094 shutdown_connections 2024-09-05T01:30:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.940+0000 7f7bf8080740 1 -- 172.21.15.129:0/461114094 wait complete. 2024-09-05T01:30:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.940+0000 7f7bf8080740 1 -- start start 2024-09-05T01:30:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.940+0000 7f7bf8080740 1 Processor -- start 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c14bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 0x556ad1c150f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c10370 0x556ad1c1c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ad1c13390 con 0x556ad1c10370 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556ad1af7290 con 0x556ad1c0a9a0 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556ad1c13510 con 0x556ad1c0b2e0 2024-09-05T01:30:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.947+0000 7f7bf8080740 10 client.0 _ll_register_callbacks cb 0x556ad1b28140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.948+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c14bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.948+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c14bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:53210/0 (socket says 172.21.15.129:53210) 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.948+0000 7f7bef7fe640 1 -- 172.21.15.129:0/20374502 learned_addr learned my addr 172.21.15.129:0/20374502 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.948+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 0x556ad1c150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.949+0000 7f7bee7fc640 1 -- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 msgr2=0x556ad1c14bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.949+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c14bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.949+0000 7f7bee7fc640 1 -- 172.21.15.129:0/20374502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c10370 msgr2=0x556ad1c1c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.949+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c10370 0x556ad1c1c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.949+0000 7f7bee7fc640 1 -- 172.21.15.129:0/20374502 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad1881210 con 0x556ad1c0b2e0 2024-09-05T01:30:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.950+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 0x556ad1c150f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009230 tx=0x7f7bd8032d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8496890adba65014 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.952+0000 7f7bd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:30:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.952+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8040070 con 0x556ad1c0b2e0 2024-09-05T01:30:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.952+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7bd8004030 con 0x556ad1c0b2e0 2024-09-05T01:30:21.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.953+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8004050 con 0x556ad1c0b2e0 2024-09-05T01:30:21.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.954+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7bd8035040 con 0x556ad1c0b2e0 2024-09-05T01:30:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.954+0000 7f7bf8080740 10 client.26359 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:21.955 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.954+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ad194d620 con 0x556ad1c0b2e0 2024-09-05T01:30:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.958+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8004050 con 0x556ad1c0b2e0 2024-09-05T01:30:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.959+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7bd80047d0 con 0x556ad1c0b2e0 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.960+0000 7f7bf8080740 10 client.26359 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.960+0000 7f7bf8080740 10 client.26359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.960+0000 7f7bf8080740 20 client.26359 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.961+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1c31830 0x556ad1c33c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.961+0000 7f7bf8080740 4 client.26359 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.961+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad1b7ba20 con 0x556ad1c31830 2024-09-05T01:30:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.962+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1c31830 0x556ad1c33c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:21.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.964+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1c31830 0x556ad1c33c20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64b17c486de8b5fd server_cookie=7b4b6236b70e4f9d in_seq=0 out_seq=0 2024-09-05T01:30:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.965+0000 7f7bd67fc640 10 client.26359 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.965+0000 7f7bd67fc640 1 -- 172.21.15.129:0/20374502 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ad1b7ba20 con 0x556ad1c31830 2024-09-05T01:30:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.965+0000 7f7bd67fc640 10 client.26359 handle_command_reply: tid=0 2024-09-05T01:30:21.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.966+0000 7f7bf8080740 1 client.26359 shutdown 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 msgr2=0x556ad1c150f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 0x556ad1c150f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009230 tx=0x7f7bd8032d30 comp rx=0 tx=0).stop 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 shutdown_connections 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1c31830 0x556ad1c33c20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c10370 0x556ad1c1c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0b2e0 0x556ad1c150f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 --2- 172.21.15.129:0/20374502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c14bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bd67fc640 0 client.26359 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:21.972 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.971+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 >> 172.21.15.129:0/20374502 conn(0x556ad1c06010 msgr2=0x556ad1c07820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:21.974 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.973+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 shutdown_connections 2024-09-05T01:30:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.975+0000 7f7bf8080740 1 -- 172.21.15.129:0/20374502 wait complete. 2024-09-05T01:30:21.975 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:21.975+0000 7f7bf8080740 20 client.26359 trim_cache size 0 max 16384 2024-09-05T01:30:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.003+0000 7f7bf8080740 1 Processor -- start 2024-09-05T01:30:22.004 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.003+0000 7f7bf8080740 1 -- start start 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c1c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bef7fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:41410/0 (socket says 172.21.15.129:41410) 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c1c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c1c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47894/0 (socket says 172.21.15.129:47894) 2024-09-05T01:30:22.005 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bef7fe640 1 -- 172.21.15.129:0/3515565398 learned_addr learned my addr 172.21.15.129:0/3515565398 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c10370 0x556ad1c07400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd00125f0 con 0x556ad1c0b2e0 2024-09-05T01:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000fec0 con 0x556ad1c10370 2024-09-05T01:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.004+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000ddc0 con 0x556ad1c0a9a0 2024-09-05T01:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c10370 0x556ad1c07400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 -- 172.21.15.129:0/3515565398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c10370 msgr2=0x556ad1c07400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c10370 0x556ad1c07400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 -- 172.21.15.129:0/3515565398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 msgr2=0x556ad1c1c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c1c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 -- 172.21.15.129:0/3515565398 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad1c13b70 con 0x556ad1c0a9a0 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.005+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7be8003cd0 tx=0x7f7be8009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c24f302fb08f9408 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.006+0000 7f7be4ff9640 1 -- 172.21.15.129:0/3515565398 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8029020 con 0x556ad1c0a9a0 2024-09-05T01:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.006+0000 7f7be4ff9640 1 -- 172.21.15.129:0/3515565398 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8003e00 con 0x556ad1c0a9a0 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.006+0000 7f7be4ff9640 1 -- 172.21.15.129:0/3515565398 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8005270 con 0x556ad1c0a9a0 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.007+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 msgr2=0x556ad1c050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.007+0000 7f7bf8080740 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7be8003cd0 tx=0x7f7be8009f50 comp rx=0 tx=0).stop 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 shutdown_connections 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c10370 0x556ad1c07400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c1c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 --2- 172.21.15.129:0/3515565398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c0a9a0 0x556ad1c050b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 >> 172.21.15.129:0/3515565398 conn(0x556ad1c06010 msgr2=0x556ad1c049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.008+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 shutdown_connections 2024-09-05T01:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.009+0000 7f7bf8080740 1 -- 172.21.15.129:0/3515565398 wait complete. 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 -- start start 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 Processor -- start 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c19ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c1b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000e380 con 0x556ad1c0b2e0 2024-09-05T01:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000f7f0 con 0x556ad1c0a9a0 2024-09-05T01:30:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd000ea50 con 0x556ad1c10370 2024-09-05T01:30:22.012 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.010+0000 7f7bf8080740 10 client.0 _ll_register_callbacks cb 0x7f7bd8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:47910/0 (socket says 172.21.15.129:47910) 2024-09-05T01:30:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 -- 172.21.15.129:0/4170073261 learned_addr learned my addr 172.21.15.129:0/4170073261 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:22.014 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 -- 172.21.15.129:0/4170073261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 msgr2=0x556ad1c1b000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c1b000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 -- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 msgr2=0x556ad1c19ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c19ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.011+0000 7f7bee7fc640 1 -- 172.21.15.129:0/4170073261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad1c1b610 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.012+0000 7f7bee7fc640 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x556ad1c16090 tx=0x7f7bd80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ceb8b718474e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.012+0000 7f7bd67fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.012+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd805c060 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.012+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8003e00 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.012+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7bd8004fc0 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7bd8063050 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 10 client.16932 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556ad1cad840 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7be8003e00 con 0x556ad1c0b2e0 2024-09-05T01:30:22.015 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7bd803c810 con 0x556ad1c0b2e0 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 10 client.16932 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 20 client.16932 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1cae170 0x556ad1cb0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 4 client.16932 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.013+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ad1b75d70 con 0x556ad1cae170 2024-09-05T01:30:22.016 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.014+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1cae170 0x556ad1cb0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.014+0000 7f7bd67fc640 10 client.16932 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:22.017 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.014+0000 7f7bef7fe640 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1cae170 0x556ad1cb0590 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dcda42cd87c4c1 server_cookie=cd28daa7b5b64dd9 in_seq=0 out_seq=0 2024-09-05T01:30:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.043+0000 7f7bd67fc640 1 -- 172.21.15.129:0/4170073261 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ad1b75d70 con 0x556ad1cae170 2024-09-05T01:30:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.043+0000 7f7bd67fc640 10 client.16932 handle_command_reply: tid=0 2024-09-05T01:30:22.044 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.044+0000 7f7bf8080740 1 client.16932 shutdown 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 msgr2=0x556ad1c19ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x556ad1c16090 tx=0x7f7bd80048f0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 shutdown_connections 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556ad1cae170 0x556ad1cb0590 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556ad1c10370 0x556ad1c1b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ad1c0b2e0 0x556ad1c19ff0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 --2- 172.21.15.129:0/4170073261 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556ad1c0a9a0 0x556ad1c19ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.046 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.045+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 >> 172.21.15.129:0/4170073261 conn(0x556ad1c06010 msgr2=0x556ad1c051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:22.047 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.046+0000 7f7bd67fc640 0 client.16932 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.047+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 shutdown_connections 2024-09-05T01:30:22.048 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.047+0000 7f7bf8080740 1 -- 172.21.15.129:0/4170073261 wait complete. 2024-09-05T01:30:22.049 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.049+0000 7f7bf8080740 20 client.16932 trim_cache size 0 max 16384 2024-09-05T01:30:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:30:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:30:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:30:22.050 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.052+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 msgr2=0x7f7bdc0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.052+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 0x7f7bdc0a5260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7be00cc7e0 tx=0x7f7be00d2860 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.052+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 msgr2=0x7f7bf019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.052+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7be000d950 tx=0x7f7be000de10 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 shutdown_connections 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7bdc0a2da0 0x7f7bdc0a5260 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7bf01025f0 0x7f7bf01a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7bf0101c30 0x7f7bf019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 --2- 172.21.15.129:0/337643571 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7bf0100a40 0x7f7bf019ea90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:22.054 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.053+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 >> 172.21.15.129:0/337643571 conn(0x7f7bf00fc1e0 msgr2=0x7f7bf0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.054+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 shutdown_connections 2024-09-05T01:30:22.055 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:22.054+0000 7f7bf52e6640 1 -- 172.21.15.129:0/337643571 wait complete. 2024-09-05T01:30:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:23 smithi129 ceph-mon[30979]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 745 op/s 2024-09-05T01:30:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:30:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:30:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:30:23.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:30:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:23 smithi194 ceph-mon[34941]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 745 op/s 2024-09-05T01:30:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:30:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:30:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:30:23.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:30:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:23 smithi183 ceph-mon[34589]: pgmap v805: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 6.4 MiB/s wr, 745 op/s 2024-09-05T01:30:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:30:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:30:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:30:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:30:25.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 800.0 s, 800.4 tps, lat 124.204 ms stddev 70.526 2024-09-05T01:30:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:25 smithi129 ceph-mon[30979]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 770 op/s 2024-09-05T01:30:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:25 smithi194 ceph-mon[34941]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 770 op/s 2024-09-05T01:30:25.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:25 smithi183 ceph-mon[34589]: pgmap v806: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.9 MiB/s wr, 770 op/s 2024-09-05T01:30:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:27 smithi129 ceph-mon[30979]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 721 op/s 2024-09-05T01:30:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:27 smithi194 ceph-mon[34941]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 721 op/s 2024-09-05T01:30:27.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:27 smithi183 ceph-mon[34589]: pgmap v807: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 721 op/s 2024-09-05T01:30:29.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:29 smithi129 ceph-mon[30979]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 851 op/s 2024-09-05T01:30:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:29 smithi194 ceph-mon[34941]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 851 op/s 2024-09-05T01:30:29.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:29 smithi183 ceph-mon[34589]: pgmap v808: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 7.7 MiB/s wr, 851 op/s 2024-09-05T01:30:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 805.0 s, 882.2 tps, lat 113.822 ms stddev 68.347 2024-09-05T01:30:31.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:31 smithi129 ceph-mon[30979]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 747 op/s 2024-09-05T01:30:31.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:31 smithi194 ceph-mon[34941]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 747 op/s 2024-09-05T01:30:31.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:31 smithi183 ceph-mon[34589]: pgmap v809: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 6.5 MiB/s wr, 747 op/s 2024-09-05T01:30:33.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:33 smithi129 ceph-mon[30979]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 816 op/s 2024-09-05T01:30:33.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:33 smithi194 ceph-mon[34941]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 816 op/s 2024-09-05T01:30:33.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:33 smithi183 ceph-mon[34589]: pgmap v810: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.3 MiB/s wr, 816 op/s 2024-09-05T01:30:35.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 810.0 s, 840.2 tps, lat 117.650 ms stddev 49.509 2024-09-05T01:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.206+0000 7f153c8b1640 1 -- 172.21.15.129:0/981188093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100b90 msgr2=0x7f1538100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.206+0000 7f153c8b1640 1 --2- 172.21.15.129:0/981188093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100b90 0x7f1538100ff0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f152c009900 tx=0x7f152c02a440 comp rx=0 tx=0).stop 2024-09-05T01:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 -- 172.21.15.129:0/981188093 shutdown_connections 2024-09-05T01:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 --2- 172.21.15.129:0/981188093 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538101530 0x7f15381019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 --2- 172.21.15.129:0/981188093 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100b90 0x7f1538100ff0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 --2- 172.21.15.129:0/981188093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538100250 0x7f1538100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 -- 172.21.15.129:0/981188093 >> 172.21.15.129:0/981188093 conn(0x7f15380fbfa0 msgr2=0x7f15380fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 -- 172.21.15.129:0/981188093 shutdown_connections 2024-09-05T01:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.207+0000 7f153c8b1640 1 -- 172.21.15.129:0/981188093 wait complete. 2024-09-05T01:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.208+0000 7f153c8b1640 1 Processor -- start 2024-09-05T01:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.208+0000 7f153c8b1640 1 -- start start 2024-09-05T01:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100250 0x7f1538195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538101530 0x7f153819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f15381052d0 con 0x7f1538101530 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1538105150 con 0x7f1538100b90 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f153c8b1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1538105450 con 0x7f1538100250 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f1536ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f1536ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40104/0 (socket says 172.21.15.129:40104) 2024-09-05T01:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f1536ffd640 1 -- 172.21.15.129:0/1741857049 learned_addr learned my addr 172.21.15.129:0/1741857049 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f15377fe640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100250 0x7f1538195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f1536ffd640 1 -- 172.21.15.129:0/1741857049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100250 msgr2=0x7f1538195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.209+0000 7f1536ffd640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100250 0x7f1538195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.210+0000 7f1536ffd640 1 -- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538101530 msgr2=0x7f153819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.210+0000 7f1536ffd640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538101530 0x7f153819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.210+0000 7f1536ffd640 1 -- 172.21.15.129:0/1741857049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819d660 con 0x7f1538100b90 2024-09-05T01:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.210+0000 7f1536ffd640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f152c004b10 tx=0x7f152c003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.210+0000 7f1534ff9640 1 -- 172.21.15.129:0/1741857049 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c040030 con 0x7f1538100b90 2024-09-05T01:30:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.211+0000 7f1534ff9640 1 -- 172.21.15.129:0/1741857049 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f152c005190 con 0x7f1538100b90 2024-09-05T01:30:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.211+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819d8f0 con 0x7f1538100b90 2024-09-05T01:30:35.213 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.211+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819deb0 con 0x7f1538100b90 2024-09-05T01:30:35.214 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.212+0000 7f1534ff9640 1 -- 172.21.15.129:0/1741857049 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c03c8b0 con 0x7f1538100b90 2024-09-05T01:30:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.215+0000 7f15408a4740 1 Processor -- start 2024-09-05T01:30:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.215+0000 7f1534ff9640 1 -- 172.21.15.129:0/1741857049 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f152c043030 con 0x7f1538100b90 2024-09-05T01:30:35.216 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.215+0000 7f15408a4740 1 -- start start 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530552df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x565305535a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5653054912f0 con 0x565305533540 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565305439270 con 0x56530552db70 2024-09-05T01:30:35.217 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15408a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56530548d2b0 con 0x56530552e4b0 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15377fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15377fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:51472/0 (socket says 172.21.15.129:51472) 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.216+0000 7f15377fe640 1 -- 172.21.15.129:0/4271545805 learned_addr learned my addr 172.21.15.129:0/4271545805 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 -- 172.21.15.129:0/4271545805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 msgr2=0x56530552df70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530552df70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 -- 172.21.15.129:0/4271545805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 msgr2=0x565305535a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x565305535a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 -- 172.21.15.129:0/4271545805 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565305438340 con 0x56530552e4b0 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f15377fe640 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f153000a4b0 tx=0x7f15300096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=404a9562c001de84 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.219 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.217+0000 7f1534ff9640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 0x7f15200a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.218+0000 7f151dffb640 1 -- 172.21.15.129:0/4271545805 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530018020 con 0x56530552e4b0 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.218+0000 7f151dffb640 1 -- 172.21.15.129:0/4271545805 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530014cc0 con 0x56530552e4b0 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.218+0000 7f151dffb640 1 -- 172.21.15.129:0/4271545805 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530013cb0 con 0x56530552e4b0 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.218+0000 7f1534ff9640 1 -- 172.21.15.129:0/1741857049 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f152c0efad0 con 0x7f1538100b90 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.219+0000 7f15408a4740 1 -- 172.21.15.129:0/4271545805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 msgr2=0x565305532f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.220 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.219+0000 7f15408a4740 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f153000a4b0 tx=0x7f15300096d0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f15408a4740 1 -- 172.21.15.129:0/4271545805 shutdown_connections 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f15408a4740 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x565305535a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f15408a4740 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x565305532f70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f15408a4740 1 --2- 172.21.15.129:0/4271545805 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530552df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f15408a4740 1 -- 172.21.15.129:0/4271545805 >> 172.21.15.129:0/4271545805 conn(0x5653055291e0 msgr2=0x5653055295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.221 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.220+0000 7f1537fff640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 0x7f15200a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.222 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.222+0000 7f1537fff640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 0x7f15200a5100 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15280058c0 tx=0x7f1528005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.224 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.223+0000 7f15408a4740 1 -- 172.21.15.129:0/4271545805 shutdown_connections 2024-09-05T01:30:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.224+0000 7f15408a4740 1 -- 172.21.15.129:0/4271545805 wait complete. 2024-09-05T01:30:35.225 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.224+0000 7f15408a4740 1 -- start start 2024-09-05T01:30:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.225+0000 7f15408a4740 1 Processor -- start 2024-09-05T01:30:35.226 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x565305538340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.228 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x56530553f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565305536510 con 0x56530552e4b0 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56530541a290 con 0x56530552db70 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x565305536690 con 0x565305533540 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.226+0000 7f15408a4740 10 client.0 _ll_register_callbacks cb 0x56530544b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.227+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.227+0000 7f1536ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x565305538340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.227+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40126/0 (socket says 172.21.15.129:40126) 2024-09-05T01:30:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.227+0000 7f15377fe640 1 -- 172.21.15.129:0/361168999 learned_addr learned my addr 172.21.15.129:0/361168999 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 -- 172.21.15.129:0/361168999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 msgr2=0x56530553f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x56530553f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 -- 172.21.15.129:0/361168999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 msgr2=0x565305538340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x565305538340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 -- 172.21.15.129:0/361168999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653051a4210 con 0x56530552db70 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f1536ffd640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x565305538340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f15377fe640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1530004a80 tx=0x7f15300094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89eb0d9025e4f4a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.228+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.229+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530014a20 con 0x56530552db70 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.229+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f153001d430 con 0x56530552db70 2024-09-05T01:30:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.229+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1530014cc0 con 0x56530552db70 2024-09-05T01:30:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.232+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1530009c80 con 0x56530552db70 2024-09-05T01:30:35.233 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f15408a4740 10 client.26510 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x565305270620 con 0x56530552db70 2024-09-05T01:30:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530014cc0 con 0x56530552db70 2024-09-05T01:30:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f1530029940 con 0x56530552db70 2024-09-05T01:30:35.234 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f15408a4740 10 client.26510 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.233+0000 7f15408a4740 10 client.26510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.234+0000 7f15408a4740 20 client.26510 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.234+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565305543d90 0x565305546210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.234+0000 7f15408a4740 4 client.26510 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:35.235 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.234+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56530549ea20 con 0x565305543d90 2024-09-05T01:30:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.234+0000 7f1536ffd640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565305543d90 0x565305546210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.235+0000 7f1516ffd640 10 client.26510 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:35.236 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.235+0000 7f1536ffd640 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565305543d90 0x565305546210 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863ffcde0ce7076c server_cookie=76e1cbaae87f229e in_seq=0 out_seq=0 2024-09-05T01:30:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.236+0000 7f1516ffd640 1 -- 172.21.15.129:0/361168999 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56530549ea20 con 0x565305543d90 2024-09-05T01:30:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.236+0000 7f1516ffd640 10 client.26510 handle_command_reply: tid=0 2024-09-05T01:30:35.237 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.236+0000 7f15408a4740 1 client.26510 shutdown 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 msgr2=0x565305537e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1530004a80 tx=0x7f15300094f0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 shutdown_connections 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x565305543d90 0x565305546210 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x56530553f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x565305538340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 --2- 172.21.15.129:0/361168999 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x565305537e00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 >> 172.21.15.129:0/361168999 conn(0x5653055291e0 msgr2=0x56530552a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f1516ffd640 0 client.26510 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 shutdown_connections 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.237+0000 7f15408a4740 1 -- 172.21.15.129:0/361168999 wait complete. 2024-09-05T01:30:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.238+0000 7f15408a4740 20 client.26510 trim_cache size 0 max 16384 2024-09-05T01:30:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.265+0000 7f15408a4740 1 Processor -- start 2024-09-05T01:30:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.265+0000 7f15408a4740 1 -- start start 2024-09-05T01:30:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.265+0000 7f15408a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.267 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15408a4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x56530553f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15408a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x565305529b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15408a4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1524012730 con 0x56530552e4b0 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40136/0 (socket says 172.21.15.129:40136) 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15377fe640 1 -- 172.21.15.129:0/692822685 learned_addr learned my addr 172.21.15.129:0/692822685 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1524010000 con 0x56530552db70 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f152400de50 con 0x565305533540 2024-09-05T01:30:35.268 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.266+0000 7f1537fff640 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x565305529b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f1536ffd640 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x56530553f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f15377fe640 1 -- 172.21.15.129:0/692822685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 msgr2=0x565305529b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f15377fe640 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x565305529b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f15377fe640 1 -- 172.21.15.129:0/692822685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 msgr2=0x56530553f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f15377fe640 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x56530553f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.267+0000 7f15377fe640 1 -- 172.21.15.129:0/692822685 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565305529000 con 0x56530552db70 2024-09-05T01:30:35.269 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.268+0000 7f15377fe640 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1530029eb0 tx=0x7f1530000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bd0a3c5ee1f6012 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.269+0000 7f151d7fa640 1 -- 172.21.15.129:0/692822685 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530018050 con 0x56530552db70 2024-09-05T01:30:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.269+0000 7f151d7fa640 1 -- 172.21.15.129:0/692822685 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530037020 con 0x56530552db70 2024-09-05T01:30:35.270 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 msgr2=0x5653055295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1530029eb0 tx=0x7f1530000f40 comp rx=0 tx=0).stop 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 shutdown_connections 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x565305533540 0x565305529b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56530552e4b0 0x56530553f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 --2- 172.21.15.129:0/692822685 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x5653055295c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 >> 172.21.15.129:0/692822685 conn(0x5653055291e0 msgr2=0x56530552f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.271 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.270+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 shutdown_connections 2024-09-05T01:30:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.271+0000 7f15408a4740 1 -- 172.21.15.129:0/692822685 wait complete. 2024-09-05T01:30:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.271+0000 7f15408a4740 1 -- start start 2024-09-05T01:30:35.272 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.271+0000 7f15408a4740 1 Processor -- start 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x56530553bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x56530553c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f152400e4c0 con 0x565305533540 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f152400f930 con 0x56530552db70 2024-09-05T01:30:35.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f152400eb90 con 0x56530552e4b0 2024-09-05T01:30:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15408a4740 10 client.0 _ll_register_callbacks cb 0x7f1530009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15377fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:40146/0 (socket says 172.21.15.129:40146) 2024-09-05T01:30:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f15377fe640 1 -- 172.21.15.129:0/1791464711 learned_addr learned my addr 172.21.15.129:0/1791464711 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:35.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.272+0000 7f1537fff640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x56530553c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 -- 172.21.15.129:0/1791464711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 msgr2=0x56530553bc60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:30:35.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x56530553bc60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 -- 172.21.15.129:0/1791464711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 msgr2=0x56530553c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x56530553c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 -- 172.21.15.129:0/1791464711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653055acae0 con 0x56530552db70 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f15377fe640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1530013fc0 tx=0x7f1530015e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55027246d35c539 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f1516ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.273+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530040070 con 0x56530552db70 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1530037020 con 0x56530552db70 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f15300096d0 con 0x56530552db70 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f153004c050 con 0x56530552db70 2024-09-05T01:30:35.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f15408a4740 10 client.26516 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5653055adc30 con 0x56530552db70 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.274+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530037020 con 0x56530552db70 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f153003c6e0 con 0x56530552db70 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 10 client.26516 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 20 client.26516 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653055ae530 0x5653055b0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 4 client.26516 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:30:35.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565305498d70 con 0x5653055ae530 2024-09-05T01:30:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.275+0000 7f1536ffd640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653055ae530 0x5653055b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.277+0000 7f1536ffd640 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653055ae530 0x5653055b0950 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b8c19fbe81489 server_cookie=98ce15aa20aaeb12 in_seq=0 out_seq=0 2024-09-05T01:30:35.278 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.277+0000 7f1516ffd640 10 client.26516 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.301+0000 7f1516ffd640 1 -- 172.21.15.129:0/1791464711 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565305498d70 con 0x5653055ae530 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f1516ffd640 10 client.26516 handle_command_reply: tid=0 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 client.26516 shutdown 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 msgr2=0x56530553f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1530013fc0 tx=0x7f1530015e20 comp rx=0 tx=0).stop 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 shutdown_connections 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5653055ae530 0x5653055b0950 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565305533540 0x56530553c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56530552e4b0 0x56530553bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 --2- 172.21.15.129:0/1791464711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56530552db70 0x56530553f1c0 secure :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1530013fc0 tx=0x7f1530015e20 comp rx=0 tx=0).stop 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 >> 172.21.15.129:0/1791464711 conn(0x5653055291e0 msgr2=0x565305527a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f1516ffd640 0 client.26516 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 shutdown_connections 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.302+0000 7f15408a4740 1 -- 172.21.15.129:0/1791464711 wait complete. 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f15408a4740 20 client.26516 trim_cache size 0 max 16384 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:30:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 msgr2=0x7f15200a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 0x7f15200a5100 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15280058c0 tx=0x7f1528005850 comp rx=0 tx=0).stop 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 msgr2=0x7f1538195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f152c004b10 tx=0x7f152c003e90 comp rx=0 tx=0).stop 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 shutdown_connections 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f15200a2c40 0x7f15200a5100 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1538101530 0x7f153819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1538100b90 0x7f1538195f80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 --2- 172.21.15.129:0/1741857049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1538100250 0x7f1538195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.303+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 >> 172.21.15.129:0/1741857049 conn(0x7f15380fbfa0 msgr2=0x7f15380fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.304+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 shutdown_connections 2024-09-05T01:30:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:35.304+0000 7f153c8b1640 1 -- 172.21.15.129:0/1741857049 wait complete. 2024-09-05T01:30:35.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:35 smithi129 ceph-mon[30979]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-09-05T01:30:35.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:35 smithi194 ceph-mon[34941]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-09-05T01:30:35.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:35 smithi183 ceph-mon[34589]: pgmap v811: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-09-05T01:30:37.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:37 smithi129 ceph-mon[30979]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 756 op/s 2024-09-05T01:30:37.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:37 smithi194 ceph-mon[34941]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 756 op/s 2024-09-05T01:30:37.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:37 smithi183 ceph-mon[34589]: pgmap v812: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.5 MiB/s wr, 756 op/s 2024-09-05T01:30:39.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:39 smithi129 ceph-mon[30979]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 911 op/s 2024-09-05T01:30:39.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:39 smithi194 ceph-mon[34941]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 911 op/s 2024-09-05T01:30:39.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:39 smithi183 ceph-mon[34589]: pgmap v813: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 8.1 MiB/s wr, 911 op/s 2024-09-05T01:30:40.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 815.0 s, 890.0 tps, lat 113.559 ms stddev 96.602 2024-09-05T01:30:41.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:41 smithi129 ceph-mon[30979]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.2 MiB/s wr, 827 op/s 2024-09-05T01:30:41.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:41 smithi194 ceph-mon[34941]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.2 MiB/s wr, 827 op/s 2024-09-05T01:30:41.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:41 smithi183 ceph-mon[34589]: pgmap v814: 129 pgs: 129 active+clean; 1.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 7.2 MiB/s wr, 827 op/s 2024-09-05T01:30:43.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:43 smithi194 ceph-mon[34941]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 878 op/s 2024-09-05T01:30:43.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:43 smithi183 ceph-mon[34589]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 878 op/s 2024-09-05T01:30:43.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:43 smithi129 ceph-mon[30979]: pgmap v815: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 7.9 MiB/s wr, 878 op/s 2024-09-05T01:30:45.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 820.0 s, 865.0 tps, lat 114.182 ms stddev 114.793 2024-09-05T01:30:45.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:45 smithi194 ceph-mon[34941]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 865 op/s 2024-09-05T01:30:45.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:45 smithi183 ceph-mon[34589]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 865 op/s 2024-09-05T01:30:45.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:45 smithi129 ceph-mon[30979]: pgmap v816: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 MiB/s wr, 865 op/s 2024-09-05T01:30:47.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:47 smithi194 ceph-mon[34941]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 863 op/s 2024-09-05T01:30:47.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:47 smithi183 ceph-mon[34589]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 863 op/s 2024-09-05T01:30:47.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:47 smithi129 ceph-mon[30979]: pgmap v817: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 7.8 MiB/s wr, 863 op/s 2024-09-05T01:30:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 -- 172.21.15.129:0/3239889283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa340093920 msgr2=0x7fa340093d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 --2- 172.21.15.129:0/3239889283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa340093920 0x7fa340093d20 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009900 tx=0x7fa33c02a410 comp rx=0 tx=0).stop 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 -- 172.21.15.129:0/3239889283 shutdown_connections 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 --2- 172.21.15.129:0/3239889283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 0x7fa34009db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 --2- 172.21.15.129:0/3239889283 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa3400968f0 0x7fa340096d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 --2- 172.21.15.129:0/3239889283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa340093920 0x7fa340093d20 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 -- 172.21.15.129:0/3239889283 >> 172.21.15.129:0/3239889283 conn(0x7fa34008f420 msgr2=0x7fa340091880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.451+0000 7fa34c87b640 1 -- 172.21.15.129:0/3239889283 shutdown_connections 2024-09-05T01:30:48.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.452+0000 7fa34c87b640 1 -- 172.21.15.129:0/3239889283 wait complete. 2024-09-05T01:30:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.452+0000 7fa34c87b640 1 Processor -- start 2024-09-05T01:30:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 -- start start 2024-09-05T01:30:48.455 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 0x7fa340132280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 0x7fa3401392b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa340096210 con 0x7fa3400968f0 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa340096090 con 0x7fa3400972b0 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa34c87b640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa340096390 con 0x7fa340093920 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:51104/0 (socket says 172.21.15.129:51104) 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa3477fe640 1 -- 172.21.15.129:0/479684195 learned_addr learned my addr 172.21.15.129:0/479684195 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.453+0000 7fa347fff640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 0x7fa3401392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa346ffd640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 0x7fa340132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.456 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 -- 172.21.15.129:0/479684195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 msgr2=0x7fa3401392b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 0x7fa3401392b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 -- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 msgr2=0x7fa340132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 0x7fa340132280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 -- 172.21.15.129:0/479684195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa340139950 con 0x7fa340093920 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa346ffd640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 0x7fa340132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa3477fe640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009850 tx=0x7fa33c003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa344ff9640 1 -- 172.21.15.129:0/479684195 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c03f070 con 0x7fa340093920 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa344ff9640 1 -- 172.21.15.129:0/479684195 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa33c031dd0 con 0x7fa340093920 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa340139be0 con 0x7fa340093920 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.454+0000 7fa344ff9640 1 -- 172.21.15.129:0/479684195 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c035030 con 0x7fa340093920 2024-09-05T01:30:48.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.455+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34013a120 con 0x7fa340093920 2024-09-05T01:30:48.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.458+0000 7fa344ff9640 1 -- 172.21.15.129:0/479684195 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa33c042070 con 0x7fa340093920 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 Processor -- start 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 -- start start 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa344ff9640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 0x7fa3380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56313f5462f0 con 0x56313f5e2b70 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56313f4ee270 con 0x56313f5e34b0 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.459+0000 7fa35087b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56313f5422b0 con 0x56313f5e8540 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa347fff640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 0x7fa3380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa344ff9640 1 -- 172.21.15.129:0/479684195 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa33c0efcc0 con 0x7fa340093920 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa347fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:36610/0 (socket says 172.21.15.129:36610) 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 -- 172.21.15.129:0/4222890112 learned_addr learned my addr 172.21.15.129:0/4222890112 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 -- 172.21.15.129:0/4222890112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 msgr2=0x56313f5eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 -- 172.21.15.129:0/4222890112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 msgr2=0x56313f5e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5e7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 -- 172.21.15.129:0/4222890112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313f4ed340 con 0x56313f5e2b70 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa347fff640 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa347fff640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 0x7fa3380a5150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa348067ea0 tx=0x7fa348050960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.460+0000 7fa3477fe640 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0c47f0 tx=0x7fa33c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc14eed394ab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.461+0000 7fa331ffb640 1 -- 172.21.15.129:0/4222890112 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c049040 con 0x56313f5e2b70 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.461+0000 7fa331ffb640 1 -- 172.21.15.129:0/4222890112 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa33c00b0b0 con 0x56313f5e2b70 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.461+0000 7fa331ffb640 1 -- 172.21.15.129:0/4222890112 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c04a260 con 0x56313f5e2b70 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 -- 172.21.15.129:0/4222890112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 msgr2=0x56313f5e2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0c47f0 tx=0x7fa33c005de0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 -- 172.21.15.129:0/4222890112 shutdown_connections 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 --2- 172.21.15.129:0/4222890112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5e2f70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 -- 172.21.15.129:0/4222890112 >> 172.21.15.129:0/4222890112 conn(0x56313f5de1e0 msgr2=0x56313f5de5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 -- 172.21.15.129:0/4222890112 shutdown_connections 2024-09-05T01:30:48.464 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.462+0000 7fa35087b740 1 -- 172.21.15.129:0/4222890112 wait complete. 2024-09-05T01:30:48.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.463+0000 7fa35087b740 1 -- start start 2024-09-05T01:30:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.463+0000 7fa35087b740 1 Processor -- start 2024-09-05T01:30:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.463+0000 7fa35087b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.463+0000 7fa35087b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.463+0000 7fa35087b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5f4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa35087b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56313f5eb2d0 con 0x56313f5e2b70 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:49052/0 (socket says 172.21.15.129:49052) 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa346ffd640 1 -- 172.21.15.129:0/725634983 learned_addr learned my addr 172.21.15.129:0/725634983 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa347fff640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa35087b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56313f4cf290 con 0x56313f5e34b0 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x56313f5eb450 con 0x56313f5e8540 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.464+0000 7fa35087b740 10 client.0 _ll_register_callbacks cb 0x56313f500140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 -- 172.21.15.129:0/725634983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 msgr2=0x56313f5f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 -- 172.21.15.129:0/725634983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 msgr2=0x56313f5ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 -- 172.21.15.129:0/725634983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56313f259210 con 0x56313f5e34b0 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa347fff640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa3477fe640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa346ffd640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa334035720 tx=0x7fa334035be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb4ce83c7f3054b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334011dd0 con 0x56313f5e34b0 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa334004020 con 0x56313f5e34b0 2024-09-05T01:30:48.470 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.465+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa33c00b0b0 con 0x56313f5e34b0 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.466+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa334004220 con 0x56313f5e34b0 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.466+0000 7fa35087b740 10 client.26528 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.466+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56313f325620 con 0x56313f5e34b0 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.466+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa33c00b0b0 con 0x56313f5e34b0 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.466+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa33403c070 con 0x56313f5e34b0 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 10 client.26528 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 10 client.26528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 20 client.26528 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5f5b40 0x56313f5f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 4 client.26528 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56313f553a20 con 0x56313f5f5b40 2024-09-05T01:30:48.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa3477fe640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5f5b40 0x56313f5f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.467+0000 7fa326ffd640 10 client.26528 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.468+0000 7fa3477fe640 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5f5b40 0x56313f5f7fc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e1598f87f872ae8 server_cookie=b0191db6c179fdb9 in_seq=0 out_seq=0 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa326ffd640 1 -- 172.21.15.129:0/725634983 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56313f553a20 con 0x56313f5f5b40 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa326ffd640 10 client.26528 handle_command_reply: tid=0 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 client.26528 shutdown 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 msgr2=0x56313f5ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa334035720 tx=0x7fa334035be0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 shutdown_connections 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5f5b40 0x56313f5f7fc0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e8540 0x56313f5f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.469+0000 7fa326ffd640 0 client.26528 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.470+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e34b0 0x56313f5ed090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.470+0000 7fa35087b740 1 --2- 172.21.15.129:0/725634983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.470+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 >> 172.21.15.129:0/725634983 conn(0x56313f5de1e0 msgr2=0x56313f5df9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.472+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 shutdown_connections 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.472+0000 7fa35087b740 1 -- 172.21.15.129:0/725634983 wait complete. 2024-09-05T01:30:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.472+0000 7fa35087b740 20 client.26528 trim_cache size 0 max 16384 2024-09-05T01:30:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 Processor -- start 2024-09-05T01:30:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 -- start start 2024-09-05T01:30:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5de1e0 0x56313f5e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5f4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.503+0000 7fa35087b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013eab0 con 0x56313f5e2b70 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa35087b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013c380 con 0x56313f5de1e0 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa35087b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013a2f0 con 0x56313f5e34b0 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:51122/0 (socket says 172.21.15.129:51122) 2024-09-05T01:30:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 -- 172.21.15.129:0/3418945425 learned_addr learned my addr 172.21.15.129:0/3418945425 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa3477fe640 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5de1e0 0x56313f5e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 -- 172.21.15.129:0/3418945425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5de1e0 msgr2=0x56313f5e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5de1e0 0x56313f5e2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 -- 172.21.15.129:0/3418945425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 msgr2=0x56313f5f4020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5f4020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.504+0000 7fa347fff640 1 -- 172.21.15.129:0/3418945425 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313f5de000 con 0x56313f5e34b0 2024-09-05T01:30:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.506+0000 7fa347fff640 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa348000b60 tx=0x7fa34807c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=164a494df4fc7310 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.507+0000 7fa3317fa640 1 -- 172.21.15.129:0/3418945425 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34807fd60 con 0x56313f5e34b0 2024-09-05T01:30:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.507+0000 7fa3317fa640 1 -- 172.21.15.129:0/3418945425 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa348093070 con 0x56313f5e34b0 2024-09-05T01:30:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.507+0000 7fa3317fa640 1 -- 172.21.15.129:0/3418945425 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480744f0 con 0x56313f5e34b0 2024-09-05T01:30:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.508+0000 7fa35087b740 1 -- 172.21.15.129:0/3418945425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 msgr2=0x56313f604cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.508+0000 7fa35087b740 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa348000b60 tx=0x7fa34807c000 comp rx=0 tx=0).stop 2024-09-05T01:30:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.509+0000 7fa35087b740 1 -- 172.21.15.129:0/3418945425 shutdown_connections 2024-09-05T01:30:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.509+0000 7fa35087b740 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5e34b0 0x56313f604cd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.509+0000 7fa35087b740 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e2b70 0x56313f5f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.509+0000 7fa35087b740 1 --2- 172.21.15.129:0/3418945425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5de1e0 0x56313f5e2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.509+0000 7fa35087b740 1 -- 172.21.15.129:0/3418945425 >> 172.21.15.129:0/3418945425 conn(0x56313f5e8060 msgr2=0x56313f6045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.511+0000 7fa35087b740 1 -- 172.21.15.129:0/3418945425 shutdown_connections 2024-09-05T01:30:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.511+0000 7fa35087b740 1 -- 172.21.15.129:0/3418945425 wait complete. 2024-09-05T01:30:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.511+0000 7fa35087b740 1 -- start start 2024-09-05T01:30:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.512+0000 7fa35087b740 1 Processor -- start 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.512+0000 7fa35087b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 0x56313f5f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.512+0000 7fa35087b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa35087b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e34b0 0x56313f5f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 0x56313f5f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 0x56313f5f02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:51132/0 (socket says 172.21.15.129:51132) 2024-09-05T01:30:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:49082/0 (socket says 172.21.15.129:49082) 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa3477fe640 1 -- 172.21.15.129:0/2437657138 learned_addr learned my addr 172.21.15.129:0/2437657138 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013a840 con 0x56313f5e34b0 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa347fff640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e34b0 0x56313f5f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013bcb0 con 0x56313f5e2b70 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa34013af10 con 0x56313f5de1e0 2024-09-05T01:30:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.513+0000 7fa35087b740 10 client.0 _ll_register_callbacks cb 0x7fa334005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.514+0000 7fa346ffd640 1 -- 172.21.15.129:0/2437657138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 msgr2=0x56313f5f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.514+0000 7fa346ffd640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 0x56313f5f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.514+0000 7fa346ffd640 1 -- 172.21.15.129:0/2437657138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e34b0 msgr2=0x56313f5f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.514+0000 7fa346ffd640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e34b0 0x56313f5f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.514+0000 7fa346ffd640 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56313f5f3d70 con 0x56313f5e2b70 2024-09-05T01:30:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.515+0000 7fa346ffd640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa334047880 tx=0x7fa334005900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455d0f2a2baa2de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.515+0000 7fa326ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:30:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.516+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334005640 con 0x56313f5e2b70 2024-09-05T01:30:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.516+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa334011bd0 con 0x56313f5e2b70 2024-09-05T01:30:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.517+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa348093070 con 0x56313f5e2b70 2024-09-05T01:30:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.517+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa334005c50 con 0x56313f5e2b70 2024-09-05T01:30:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.517+0000 7fa35087b740 10 client.26534 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.517+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56313f662fd0 con 0x56313f5e2b70 2024-09-05T01:30:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.517+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa334011df0 con 0x56313f5e2b70 2024-09-05T01:30:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa33403c040 con 0x56313f5e2b70 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 10 client.26534 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 10 client.26534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 20 client.26534 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5e8540 0x56313f665990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 4 client.26534 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:30:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.518+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa348093070 con 0x56313f5e8540 2024-09-05T01:30:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.519+0000 7fa3477fe640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5e8540 0x56313f665990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.520+0000 7fa3477fe640 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5e8540 0x56313f665990 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6688939e870678bc server_cookie=8b005f805efcb6c5 in_seq=0 out_seq=0 2024-09-05T01:30:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.520+0000 7fa326ffd640 10 client.26534 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.543+0000 7fa326ffd640 1 -- 172.21.15.129:0/2437657138 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa348093070 con 0x56313f5e8540 2024-09-05T01:30:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.543+0000 7fa326ffd640 10 client.26534 handle_command_reply: tid=0 2024-09-05T01:30:48.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.543+0000 7fa35087b740 1 client.26534 shutdown 2024-09-05T01:30:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.547+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 msgr2=0x56313f5f0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.547+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa334047880 tx=0x7fa334005900 comp rx=0 tx=0).stop 2024-09-05T01:30:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 shutdown_connections 2024-09-05T01:30:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56313f5e8540 0x56313f665990 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56313f5e34b0 0x56313f5f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56313f5e2b70 0x56313f5f0800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 --2- 172.21.15.129:0/2437657138 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x56313f5de1e0 0x56313f5f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.549+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 >> 172.21.15.129:0/2437657138 conn(0x56313f5e8060 msgr2=0x56313f6045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.551+0000 7fa326ffd640 0 client.26534 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.553+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 shutdown_connections 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.553+0000 7fa35087b740 1 -- 172.21.15.129:0/2437657138 wait complete. 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.553+0000 7fa35087b740 20 client.26534 trim_cache size 0 max 16384 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:30:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:30:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:30:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:30:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 msgr2=0x7fa3380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 0x7fa3380a5150 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa348067ea0 tx=0x7fa348050960 comp rx=0 tx=0).stop 2024-09-05T01:30:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 msgr2=0x7fa340131d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa33c009850 tx=0x7fa33c003ed0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 shutdown_connections 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa3380a2c90 0x7fa3380a5150 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.554+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa3400972b0 0x7fa3401392b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.555+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3400968f0 0x7fa340132280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.555+0000 7fa34c87b640 1 --2- 172.21.15.129:0/479684195 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa340093920 0x7fa340131d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.555+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 >> 172.21.15.129:0/479684195 conn(0x7fa34008f420 msgr2=0x7fa3400a16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.555+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 shutdown_connections 2024-09-05T01:30:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:30:48.555+0000 7fa34c87b640 1 -- 172.21.15.129:0/479684195 wait complete. 2024-09-05T01:30:49.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:49 smithi183 ceph-mon[34589]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.5 MiB/s wr, 944 op/s 2024-09-05T01:30:49.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:49 smithi129 ceph-mon[30979]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.5 MiB/s wr, 944 op/s 2024-09-05T01:30:49.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:49 smithi194 ceph-mon[34941]: pgmap v818: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 8.5 MiB/s wr, 944 op/s 2024-09-05T01:30:50.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 825.0 s, 859.8 tps, lat 117.329 ms stddev 120.527 2024-09-05T01:30:51.179 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.6 2024-09-05T01:30:51.180 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:30:51.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:51 smithi129 ceph-mon[30979]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 792 op/s 2024-09-05T01:30:51.622 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:30:51.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:51 smithi183 ceph-mon[34589]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 792 op/s 2024-09-05T01:30:51.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:51 smithi194 ceph-mon[34941]: pgmap v819: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 7.0 MiB/s wr, 792 op/s 2024-09-05T01:30:53.510 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:53 smithi129 ceph-mon[30979]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.8 MiB/s wr, 847 op/s 2024-09-05T01:30:53.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:53 smithi183 ceph-mon[34589]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.8 MiB/s wr, 847 op/s 2024-09-05T01:30:53.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:53 smithi194 ceph-mon[34941]: pgmap v820: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 7.8 MiB/s wr, 847 op/s 2024-09-05T01:30:55.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 830.0 s, 803.2 tps, lat 124.482 ms stddev 123.194 2024-09-05T01:30:55.599 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:55 smithi129 ceph-mon[30979]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 807 op/s 2024-09-05T01:30:55.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:55 smithi183 ceph-mon[34589]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 807 op/s 2024-09-05T01:30:55.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:55 smithi194 ceph-mon[34941]: pgmap v821: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 7.2 MiB/s wr, 807 op/s 2024-09-05T01:30:55.883 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.731+0000 7f5707d45640 1 -- 172.21.15.129:0/2099229808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700109830 msgr2=0x7f5700109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.731+0000 7f5707d45640 1 --2- 172.21.15.129:0/2099229808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700109830 0x7f5700109c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f56fc009900 tx=0x7f56fc02a440 comp rx=0 tx=0).stop 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.732+0000 7f5707d45640 1 -- 172.21.15.129:0/2099229808 shutdown_connections 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.732+0000 7f5707d45640 1 --2- 172.21.15.129:0/2099229808 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f570011b140 0x7f570011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.732+0000 7f5707d45640 1 --2- 172.21.15.129:0/2099229808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5700109830 0x7f5700109c90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.732+0000 7f5707d45640 1 --2- 172.21.15.129:0/2099229808 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f5700110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.732+0000 7f5707d45640 1 -- 172.21.15.129:0/2099229808 >> 172.21.15.129:0/2099229808 conn(0x7f570006e140 msgr2=0x7f570006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.737+0000 7f5707d45640 1 -- 172.21.15.129:0/2099229808 shutdown_connections 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.737+0000 7f5707d45640 1 -- 172.21.15.129:0/2099229808 wait complete. 2024-09-05T01:30:55.884 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.737+0000 7f5707d45640 1 Processor -- start 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 -- start start 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5700109830 0x7f5700114830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f570011b140 0x7f5700114d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f57001111e0 con 0x7f5700109830 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111060 con 0x7f570011b140 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5707d45640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111360 con 0x7f5700109450 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5705ae1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5705ae1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:53814/0 (socket says 172.21.15.129:53814) 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f5705ae1640 1 -- 172.21.15.129:0/2509209616 learned_addr learned my addr 172.21.15.129:0/2509209616 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.740+0000 7f57062e2640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f570011b140 0x7f5700114d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:55.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 -- 172.21.15.129:0/2509209616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f570011b140 msgr2=0x7f5700114d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f570011b140 0x7f5700114d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 -- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5700109830 msgr2=0x7f5700114830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5700109830 0x7f5700114830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 -- 172.21.15.129:0/2509209616 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57001153b0 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f5705ae1640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56f800cce0 tx=0x7f56f800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.741+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8004b20 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.742+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56f8004cc0 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.742+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f800f6b0 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.742+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57001bc130 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.742+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001bc4d0 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.745+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f56f800f850 con 0x7f5700109450 2024-09-05T01:30:55.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.746+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5700109c90 con 0x7f5700109450 2024-09-05T01:30:55.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.746+0000 7f56f6ffd640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 0x7f56ec0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:55.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.747+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f56f80cf4e0 con 0x7f5700109450 2024-09-05T01:30:55.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.747+0000 7f57052e0640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 0x7f56ec0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:55.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.749+0000 7f57052e0640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 0x7f56ec0a5250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0096c0 tx=0x7f56fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:55.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:55.752+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56f8014030 con 0x7f5700109450 2024-09-05T01:30:56.135 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.134+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f570010d5c0 con 0x7f5700109450 2024-09-05T01:30:56.137 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.136+0000 7f56f6ffd640 1 -- 172.21.15.129:0/2509209616 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f56f809bfd0 con 0x7f5700109450 2024-09-05T01:30:56.137 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:30:56.138 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:30:56.138 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:30:56.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.140+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 msgr2=0x7f56ec0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:56.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.140+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 0x7f56ec0a5250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f56fc0096c0 tx=0x7f56fc000bc0 comp rx=0 tx=0).stop 2024-09-05T01:30:56.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.140+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 msgr2=0x7f570011a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:56.141 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.140+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56f800cce0 tx=0x7f56f800a620 comp rx=0 tx=0).stop 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 shutdown_connections 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56ec0a2d90 0x7f56ec0a5250 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f570011b140 0x7f5700114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5700109830 0x7f5700114830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 --2- 172.21.15.129:0/2509209616 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5700109450 0x7f570011a760 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 >> 172.21.15.129:0/2509209616 conn(0x7f570006e140 msgr2=0x7f5700070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 shutdown_connections 2024-09-05T01:30:56.142 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:56.141+0000 7f5707d45640 1 -- 172.21.15.129:0/2509209616 wait complete. 2024-09-05T01:30:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:56 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2509209616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:56.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:56 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2509209616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:56.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:56 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2509209616' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:30:56.876 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 131.6 --await-for 45.0' 2024-09-05T01:30:56.876 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 131.6 --await-for 45.0 2024-09-05T01:30:57.355 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:30:57.658 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:57 smithi129 ceph-mon[30979]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 742 op/s 2024-09-05T01:30:57.660 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:57 smithi183 ceph-mon[34589]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 742 op/s 2024-09-05T01:30:57.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:57 smithi194 ceph-mon[34941]: pgmap v822: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 742 op/s 2024-09-05T01:30:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.517+0000 7f62e9ebb640 1 -- 172.21.15.129:0/4194573614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 msgr2=0x7f62e406df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.517+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/4194573614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e406df80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f62d8009860 tx=0x7f62d802a340 comp rx=0 tx=0).stop 2024-09-05T01:30:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 -- 172.21.15.129:0/4194573614 shutdown_connections 2024-09-05T01:30:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/4194573614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e411b210 0x7f62e411d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/4194573614 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e406e4c0 0x7f62e411acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/4194573614 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e406df80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 -- 172.21.15.129:0/4194573614 >> 172.21.15.129:0/4194573614 conn(0x7f62e406cdc0 msgr2=0x7f62e406d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.518+0000 7f62e9ebb640 1 -- 172.21.15.129:0/4194573614 shutdown_connections 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.519+0000 7f62e9ebb640 1 -- 172.21.15.129:0/4194573614 wait complete. 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 Processor -- start 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 -- start start 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e411a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e406e4c0 0x7f62e41169a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410ed10 con 0x7f62e406e4c0 2024-09-05T01:30:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410eb90 con 0x7f62e411b210 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.520+0000 7f62e9ebb640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62e410ee90 con 0x7f62e406dba0 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34412/0 (socket says 172.21.15.129:34412) 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 -- 172.21.15.129:0/2120789966 learned_addr learned my addr 172.21.15.129:0/2120789966 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e411a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e406e4c0 0x7f62e41169a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 -- 172.21.15.129:0/2120789966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 msgr2=0x7f62e411a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e411a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 -- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e406e4c0 msgr2=0x7f62e41169a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e406e4c0 0x7f62e41169a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 -- 172.21.15.129:0/2120789966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e4117610 con 0x7f62e411b210 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.521+0000 7f62e3fff640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01fd90 tx=0x7f62dc01d750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.522+0000 7f62e0ff9640 1 -- 172.21.15.129:0/2120789966 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc01de60 con 0x7f62e411b210 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.522+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e41ab140 con 0x7f62e411b210 2024-09-05T01:30:58.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.522+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e41ab540 con 0x7f62e411b210 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.522+0000 7f62e0ff9640 1 -- 172.21.15.129:0/2120789966 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62dc003970 con 0x7f62e411b210 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.522+0000 7f62e0ff9640 1 -- 172.21.15.129:0/2120789966 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0047a0 con 0x7f62e411b210 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.525+0000 7f62e0ff9640 1 -- 172.21.15.129:0/2120789966 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f62dc02e020 con 0x7f62e411b210 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.525+0000 7f62ec294740 1 Processor -- start 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 -- start start 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060700b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x55806070dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580606662a0 con 0x55806070b940 2024-09-05T01:30:58.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55806060db10 con 0x558060701080 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62ec294740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x558060662260 con 0x558060700760 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62e37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62e37fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34418/0 (socket says 172.21.15.129:34418) 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.526+0000 7f62e37fe640 1 -- 172.21.15.129:0/2670179696 learned_addr learned my addr 172.21.15.129:0/2670179696 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e0ff9640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 0x7f62cc0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e3fff640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 0x7f62cc0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060700b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x55806070dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e37fe640 1 -- 172.21.15.129:0/2670179696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 msgr2=0x558060700b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060700b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.618 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e37fe640 1 -- 172.21.15.129:0/2670179696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 msgr2=0x55806070dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x55806070dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e0ff9640 1 -- 172.21.15.129:0/2120789966 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f62dc0dfe90 con 0x7f62e411b210 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.527+0000 7f62e37fe640 1 -- 172.21.15.129:0/2670179696 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580603d7b90 con 0x558060701080 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.528+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x55806070dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.528+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62d802a930 tx=0x7f62d8009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1750553858d4e566 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.528+0000 7f62c9ffb640 1 -- 172.21.15.129:0/2670179696 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8042020 con 0x558060701080 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.528+0000 7f62c9ffb640 1 -- 172.21.15.129:0/2670179696 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d8038040 con 0x558060701080 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.528+0000 7f62c9ffb640 1 -- 172.21.15.129:0/2670179696 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d8037040 con 0x558060701080 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.529+0000 7f62ec294740 1 -- 172.21.15.129:0/2670179696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 msgr2=0x55806070b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.529+0000 7f62ec294740 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f62d802a930 tx=0x7f62d8009520 comp rx=0 tx=0).stop 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62ec294740 1 -- 172.21.15.129:0/2670179696 shutdown_connections 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62ec294740 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x55806070dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62ec294740 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x55806070b400 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62ec294740 1 --2- 172.21.15.129:0/2670179696 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060700b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62ec294740 1 -- 172.21.15.129:0/2670179696 >> 172.21.15.129:0/2670179696 conn(0x5580606fccf0 msgr2=0x5580606fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.530+0000 7f62e3fff640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 0x7f62cc0a5100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01dc50 tx=0x7f62dc078450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.531+0000 7f62ec294740 1 -- 172.21.15.129:0/2670179696 shutdown_connections 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.531+0000 7f62ec294740 1 -- 172.21.15.129:0/2670179696 wait complete. 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.531+0000 7f62ec294740 1 -- start start 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.531+0000 7f62ec294740 1 Processor -- start 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x558060708cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x558060709200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55806070e510 con 0x55806070b940 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580605ee6e0 con 0x558060701080 2024-09-05T01:30:58.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55806070e690 con 0x558060700760 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62ec294740 10 client.0 _ll_register_callbacks cb 0x558060612a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:53856/0 (socket says 172.21.15.129:53856) 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 -- 172.21.15.129:0/86924287 learned_addr learned my addr 172.21.15.129:0/86924287 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e37fe640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x558060708cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 -- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 msgr2=0x558060708cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x558060708cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 -- 172.21.15.129:0/86924287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 msgr2=0x558060709200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.532+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x558060709200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62e2ffd640 1 -- 172.21.15.129:0/86924287 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558060717a10 con 0x558060700760 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62d4012820 tx=0x7f62d4012ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e52ca97a1af3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:30:58.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4011af0 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d8038040 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f62d4011c90 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.533+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f62d4011e90 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.534+0000 7f62ec294740 10 client.26407 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.534+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55806071a340 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.534+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d8038040 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f62d400f590 con 0x558060700760 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 10 client.26407 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 10 client.26407 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 20 client.26407 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55806071b800 0x55806071dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 4 client.26407 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:30:58.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580606739d0 con 0x55806071b800 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.535+0000 7f62e37fe640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55806071b800 0x55806071dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.536+0000 7f62e37fe640 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55806071b800 0x55806071dc50 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440bc1632f5cdc03 server_cookie=83e164bff5b7b252 in_seq=0 out_seq=0 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.537+0000 7f62c2ffd640 10 client.26407 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.537+0000 7f62c2ffd640 1 -- 172.21.15.129:0/86924287 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580606739d0 con 0x55806071b800 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.537+0000 7f62c2ffd640 10 client.26407 handle_command_reply: tid=0 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.537+0000 7f62ec294740 1 client.26407 shutdown 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 msgr2=0x558060708780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62d4012820 tx=0x7f62d4012ce0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 shutdown_connections 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55806071b800 0x55806071dc50 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55806070b940 0x558060709200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060701080 0x558060708cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 --2- 172.21.15.129:0/86924287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060700760 0x558060708780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 >> 172.21.15.129:0/86924287 conn(0x5580606fccf0 msgr2=0x5580606fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62c2ffd640 0 client.26407 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 shutdown_connections 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 1 -- 172.21.15.129:0/86924287 wait complete. 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.538+0000 7f62ec294740 20 client.26407 trim_cache size 0 max 16384 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.571+0000 7f62ec294740 1 Processor -- start 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.571+0000 7f62ec294740 1 -- start start 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 0x5580607031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060701080 0x558060716f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 0x558060717440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0012730 con 0x558060700760 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62d0010000 con 0x55806070b940 2024-09-05T01:30:58.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62ec294740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62d000de50 con 0x558060701080 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 0x5580607031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 0x5580607031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58298/0 (socket says 172.21.15.129:58298) 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2425804449 learned_addr learned my addr 172.21.15.129:0/2425804449 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.572+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 0x558060717440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.573+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2425804449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060701080 msgr2=0x558060716f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.573+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060701080 0x558060716f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.573+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2425804449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 msgr2=0x5580607031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.573+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 0x5580607031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.573+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2425804449 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580606fcb20 con 0x55806070b940 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.575+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 0x558060717440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62d400f9e0 tx=0x7f62d4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d17372411ed9b11f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.575+0000 7f62c97fa640 1 -- 172.21.15.129:0/2425804449 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4011d50 con 0x55806070b940 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.575+0000 7f62c97fa640 1 -- 172.21.15.129:0/2425804449 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d402d450 con 0x55806070b940 2024-09-05T01:30:58.625 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.575+0000 7f62c97fa640 1 -- 172.21.15.129:0/2425804449 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d401c680 con 0x55806070b940 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.576+0000 7f62ec294740 1 -- 172.21.15.129:0/2425804449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 msgr2=0x558060717440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 0x558060717440 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f62d400f9e0 tx=0x7f62d4000bc0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 -- 172.21.15.129:0/2425804449 shutdown_connections 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55806070b940 0x558060717440 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x558060701080 0x558060716f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 --2- 172.21.15.129:0/2425804449 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060700760 0x5580607031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.577+0000 7f62ec294740 1 -- 172.21.15.129:0/2425804449 >> 172.21.15.129:0/2425804449 conn(0x5580606fccf0 msgr2=0x5580607167f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.578+0000 7f62ec294740 1 -- 172.21.15.129:0/2425804449 shutdown_connections 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.579+0000 7f62ec294740 1 -- 172.21.15.129:0/2425804449 wait complete. 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.579+0000 7f62ec294740 1 -- start start 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.580+0000 7f62ec294740 1 Processor -- start 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.626 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060701080 0x5580607102d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55806070b940 0x558060785530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f62d000e4c0 con 0x558060701080 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62d000f930 con 0x558060700760 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f62d000eb90 con 0x55806070b940 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62ec294740 10 client.0 _ll_register_callbacks cb 0x7f62d4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62e2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:34442/0 (socket says 172.21.15.129:34442) 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.581+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2007274905 learned_addr learned my addr 172.21.15.129:0/2007274905 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55806070b940 0x558060785530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2007274905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55806070b940 msgr2=0x558060785530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55806070b940 0x558060785530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2007274905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060701080 msgr2=0x5580607102d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:30:58.627 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060701080 0x5580607102d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 -- 172.21.15.129:0/2007274905 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580606f8260 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.582+0000 7f62e2ffd640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006010 tx=0x7f62d4012ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35015b17840214f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.583+0000 7f62c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.583+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d4004250 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.583+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f62d4036070 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.585+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62d402d450 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.586+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f62d40043f0 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.586+0000 7f62ec294740 10 client.26558 fetch_fsmap learned FSMap version 34 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.586+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558060786ac0 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62d402dab0 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f62d403b020 con 0x558060700760 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 10 client.26558 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:30:58.628 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 10 client.26558 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 20 client.26558 populate_metadata read hostname 'smithi129' 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580607871a0 0x5580607895e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 4 client.26558 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.6, "await_for": 45.0}] 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.6, "await_for": 45.0}) -- 0x7f62d402d450 con 0x5580607871a0 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.587+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580607871a0 0x5580607895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.588+0000 7f62e37fe640 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580607871a0 0x5580607895e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=575db4ab348b5c4a server_cookie=5f1740a4e2ce86d5 in_seq=0 out_seq=0 2024-09-05T01:30:58.629 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.589+0000 7f62c2ffd640 10 client.26558 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:58.993 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62c2ffd640 1 -- 172.21.15.129:0/2007274905 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f62d402d450 con 0x5580607871a0 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62c2ffd640 10 client.26558 handle_command_reply: tid=0 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 client.26558 shutdown 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 msgr2=0x55806070fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f62d4006010 tx=0x7f62d4012ce0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 shutdown_connections 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580607871a0 0x5580607895e0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55806070b940 0x558060785530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558060701080 0x5580607102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62c2ffd640 0 client.26558 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 --2- 172.21.15.129:0/2007274905 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558060700760 0x55806070fd90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 >> 172.21.15.129:0/2007274905 conn(0x5580606fccf0 msgr2=0x558060717010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.994 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 shutdown_connections 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 1 -- 172.21.15.129:0/2007274905 wait complete. 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.991+0000 7f62ec294740 20 client.26558 trim_cache size 0 max 16384 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 58, 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "43362322": { 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 58, 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:30:58.995 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 131.6, 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:30:58.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:30:58.997 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 msgr2=0x7f62cc0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 0x7f62cc0a5100 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01dc50 tx=0x7f62dc078450 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 msgr2=0x7f62e4116ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f62dc01fd90 tx=0x7f62dc01d750 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 shutdown_connections 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f62cc0a2c40 0x7f62cc0a5100 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62e411b210 0x7f62e4116ee0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f62e406e4c0 0x7f62e41169a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 --2- 172.21.15.129:0/2120789966 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f62e406dba0 0x7f62e411a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 >> 172.21.15.129:0/2120789966 conn(0x7f62e406cdc0 msgr2=0x7f62e411b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 shutdown_connections 2024-09-05T01:30:58.998 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:30:58.992+0000 7f62e9ebb640 1 -- 172.21.15.129:0/2120789966 wait complete. 2024-09-05T01:30:59.554 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:30:59 smithi129 ceph-mon[30979]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 827 op/s 2024-09-05T01:30:59.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:30:59 smithi183 ceph-mon[34589]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 827 op/s 2024-09-05T01:30:59.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:30:59 smithi194 ceph-mon[34941]: pgmap v823: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.8 MiB/s wr, 827 op/s 2024-09-05T01:30:59.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '43362322', quiesce took 8.7 seconds. Will release after: 2.9000000000000004 2024-09-05T01:31:00.208 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 835.0 s, 559.0 tps, lat 123.055 ms stddev 51.846 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 -- 172.21.15.129:0/2806849772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c100b70 msgr2=0x7f752c100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 --2- 172.21.15.129:0/2806849772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c100b70 0x7f752c100ff0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7528009900 tx=0x7f752802a410 comp rx=0 tx=0).stop 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 -- 172.21.15.129:0/2806849772 shutdown_connections 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 --2- 172.21.15.129:0/2806849772 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c101530 0x7f752c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 --2- 172.21.15.129:0/2806849772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c100b70 0x7f752c100ff0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 --2- 172.21.15.129:0/2806849772 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c0ffe10 0x7f752c100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 -- 172.21.15.129:0/2806849772 >> 172.21.15.129:0/2806849772 conn(0x7f752c0fb970 msgr2=0x7f752c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 -- 172.21.15.129:0/2806849772 shutdown_connections 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.657+0000 7f75333d6640 1 -- 172.21.15.129:0/2806849772 wait complete. 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 Processor -- start 2024-09-05T01:31:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 -- start start 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c0ffe10 0x7f752c199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 0x7f752c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c10f8f0 con 0x7f752c101530 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f752c10f770 con 0x7f752c0ffe10 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f75333d6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f752c10fa70 con 0x7f752c100b70 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f7531973640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f7531973640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58326/0 (socket says 172.21.15.129:58326) 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.658+0000 7f7531973640 1 -- 172.21.15.129:0/2128233006 learned_addr learned my addr 172.21.15.129:0/2128233006 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7530971640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 0x7f752c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531172640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c0ffe10 0x7f752c199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 -- 172.21.15.129:0/2128233006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 msgr2=0x7f752c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 0x7f752c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 -- 172.21.15.129:0/2128233006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c0ffe10 msgr2=0x7f752c199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c0ffe10 0x7f752c199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 -- 172.21.15.129:0/2128233006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c1a1b80 con 0x7f752c101530 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7530971640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 0x7f752c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f7531973640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f752400cad0 tx=0x7f752400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f75227fc640 1 -- 172.21.15.129:0/2128233006 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400ae20 con 0x7f752c101530 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f752c1a1e70 con 0x7f752c101530 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f75227fc640 1 -- 172.21.15.129:0/2128233006 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7524005020 con 0x7f752c101530 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f75227fc640 1 -- 172.21.15.129:0/2128233006 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752400f6b0 con 0x7f752c101530 2024-09-05T01:31:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.659+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f752c1a23b0 con 0x7f752c101530 2024-09-05T01:31:01.665 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:01 smithi183 ceph-mon[34589]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 758 op/s 2024-09-05T01:31:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.661+0000 7f75227fc640 1 -- 172.21.15.129:0/2128233006 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f75240051d0 con 0x7f752c101530 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 Processor -- start 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 -- start start 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0ef7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0efa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615c0e562f0 con 0x5615c0ef29a0 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615c0dfe270 con 0x5615c0ef32e0 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.662+0000 7f753624a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5615c0e522b0 con 0x5615c0ef8370 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58332/0 (socket says 172.21.15.129:58332) 2024-09-05T01:31:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 -- 172.21.15.129:0/18923636 learned_addr learned my addr 172.21.15.129:0/18923636 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f75227fc640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 0x7f75180a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0efa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7531172640 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0ef7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7531973640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 0x7f75180a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 -- 172.21.15.129:0/18923636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 msgr2=0x5615c0efa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0efa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 -- 172.21.15.129:0/18923636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 msgr2=0x5615c0ef7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0ef7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 -- 172.21.15.129:0/18923636 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c0dfd340 con 0x5615c0ef29a0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f75227fc640 1 -- 172.21.15.129:0/2128233006 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f75240cf640 con 0x7f752c101530 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7530971640 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f7528005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164db34035302aad server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f75177fe640 1 -- 172.21.15.129:0/18923636 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528035050 con 0x5615c0ef29a0 2024-09-05T01:31:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f7531973640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 0x7f75180a50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f752409b730 tx=0x7f75240963e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f75177fe640 1 -- 172.21.15.129:0/18923636 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7528031880 con 0x5615c0ef29a0 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.663+0000 7f75177fe640 1 -- 172.21.15.129:0/18923636 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528004920 con 0x5615c0ef29a0 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 -- 172.21.15.129:0/18923636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 msgr2=0x5615c0ef2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7528006010 tx=0x7f7528005e50 comp rx=0 tx=0).stop 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 -- 172.21.15.129:0/18923636 shutdown_connections 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0efa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0ef7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 --2- 172.21.15.129:0/18923636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0ef2da0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 -- 172.21.15.129:0/18923636 >> 172.21.15.129:0/18923636 conn(0x5615c0eee010 msgr2=0x5615c0eee410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 -- 172.21.15.129:0/18923636 shutdown_connections 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.664+0000 7f753624a740 1 -- 172.21.15.129:0/18923636 wait complete. 2024-09-05T01:31:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 -- start start 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 Processor -- start 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0efd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f04140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615c0efb2e0 con 0x5615c0ef29a0 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5615c0ddf290 con 0x5615c0ef32e0 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5615c0efb460 con 0x5615c0ef8370 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f753624a740 10 client.0 _ll_register_callbacks cb 0x5615c0e10140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58342/0 (socket says 172.21.15.129:58342) 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 -- 172.21.15.129:0/593636701 learned_addr learned my addr 172.21.15.129:0/593636701 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 -- 172.21.15.129:0/593636701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 msgr2=0x5615c0f04140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f04140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 -- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 msgr2=0x5615c0efd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0efd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.665+0000 7f7530971640 1 -- 172.21.15.129:0/593636701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c0b69210 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7530971640 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7528004630 tx=0x7f75280055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daae40a76177d7b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752804c020 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7528003a30 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7528031880 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7528035040 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 10 client.16977 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5615c0c35620 con 0x5615c0ef29a0 2024-09-05T01:31:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7528031880 con 0x5615c0ef29a0 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7528039410 con 0x5615c0ef29a0 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 10 client.16977 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 10 client.16977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 20 client.16977 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f08440 0x5615c0f0a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 4 client.16977 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.666+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615c0e63a20 con 0x5615c0f08440 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.667+0000 7f7531172640 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f08440 0x5615c0f0a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.667+0000 7f7514ff9640 10 client.16977 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.667+0000 7f7531172640 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f08440 0x5615c0f0a830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58947b2915f2f64f server_cookie=5960e5b0eb8d5a71 in_seq=0 out_seq=0 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f7514ff9640 1 -- 172.21.15.129:0/593636701 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615c0e63a20 con 0x5615c0f08440 2024-09-05T01:31:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f7514ff9640 10 client.16977 handle_command_reply: tid=0 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 client.16977 shutdown 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 msgr2=0x5615c0efcbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f7528004630 tx=0x7f75280055f0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 shutdown_connections 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f08440 0x5615c0f0a830 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f04140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f7514ff9640 0 client.16977 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef32e0 0x5615c0efd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 --2- 172.21.15.129:0/593636701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efcbc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 >> 172.21.15.129:0/593636701 conn(0x5615c0eee010 msgr2=0x5615c0eef820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 shutdown_connections 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 1 -- 172.21.15.129:0/593636701 wait complete. 2024-09-05T01:31:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.668+0000 7f753624a740 20 client.16977 trim_cache size 0 max 16384 2024-09-05T01:31:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.682+0000 7f753624a740 1 Processor -- start 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 -- start start 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef32e0 0x5615c0f04100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef8370 0x5615c0eed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1af140 con 0x5615c0ef29a0 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1aca10 con 0x5615c0ef8370 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f753624a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1aa860 con 0x5615c0ef32e0 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58344/0 (socket says 172.21.15.129:58344) 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 -- 172.21.15.129:0/3619533606 learned_addr learned my addr 172.21.15.129:0/3619533606 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7531172640 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef32e0 0x5615c0f04100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 -- 172.21.15.129:0/3619533606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef32e0 msgr2=0x5615c0f04100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef32e0 0x5615c0f04100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 -- 172.21.15.129:0/3619533606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef8370 msgr2=0x5615c0eed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef8370 0x5615c0eed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 -- 172.21.15.129:0/3619533606 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615c0f0aba0 con 0x5615c0ef29a0 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7530971640 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7528039860 tx=0x7f7528000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d223fdad293d56e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7516ffd640 1 -- 172.21.15.129:0/3619533606 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528060070 con 0x5615c0ef29a0 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7516ffd640 1 -- 172.21.15.129:0/3619533606 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7528041070 con 0x5615c0ef29a0 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.683+0000 7f7516ffd640 1 -- 172.21.15.129:0/3619533606 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752804c040 con 0x5615c0ef29a0 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- 172.21.15.129:0/3619533606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 msgr2=0x5615c0efd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7528039860 tx=0x7f7528000bc0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- 172.21.15.129:0/3619533606 shutdown_connections 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef8370 0x5615c0eed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef32e0 0x5615c0f04100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 --2- 172.21.15.129:0/3619533606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef29a0 0x5615c0efd190 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- 172.21.15.129:0/3619533606 >> 172.21.15.129:0/3619533606 conn(0x5615c0eee010 msgr2=0x5615c0eeeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- 172.21.15.129:0/3619533606 shutdown_connections 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- 172.21.15.129:0/3619533606 wait complete. 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 -- start start 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.684+0000 7f753624a740 1 Processor -- start 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef29a0 0x5615c0f01680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1aaed0 con 0x5615c0ef32e0 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1ac340 con 0x5615c0ef29a0 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f752c1ab5a0 con 0x5615c0ef8370 2024-09-05T01:31:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 10 client.0 _ll_register_callbacks cb 0x7f7528032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:58360/0 (socket says 172.21.15.129:58360) 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 -- 172.21.15.129:0/575833135 learned_addr learned my addr 172.21.15.129:0/575833135 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7530971640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 -- 172.21.15.129:0/575833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 msgr2=0x5615c0f02bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f02bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 -- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef29a0 msgr2=0x5615c0f01680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef29a0 0x5615c0f01680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 -- 172.21.15.129:0/575833135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c0f031e0 con 0x5615c0ef32e0 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7530971640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f02bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7531172640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f751c006e10 tx=0x7f751c00bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ded43a8303538f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7514ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751c0156f0 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f751c015890 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.685+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7528041070 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f751c02c050 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 10 client.16983 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5615c0f730b0 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7528041070 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f751c01ece0 con 0x5615c0ef32e0 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 10 client.16983 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 20 client.16983 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f73880 0x5615c0f75ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 4 client.16983 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:31:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615c0e5dd70 con 0x5615c0f73880 2024-09-05T01:31:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.686+0000 7f7530971640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f73880 0x5615c0f75ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.687+0000 7f7514ff9640 10 client.16983 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.687+0000 7f7530971640 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f73880 0x5615c0f75ca0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f50d21cd0e1b79c1 server_cookie=73f4e638569b3505 in_seq=0 out_seq=0 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f7514ff9640 1 -- 172.21.15.129:0/575833135 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615c0e5dd70 con 0x5615c0f73880 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f7514ff9640 10 client.16983 handle_command_reply: tid=0 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 client.16983 shutdown 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 msgr2=0x5615c0f01bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f751c006e10 tx=0x7f751c00bcc0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 shutdown_connections 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5615c0f73880 0x5615c0f75ca0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5615c0ef8370 0x5615c0f02bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615c0ef32e0 0x5615c0f01bc0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f7514ff9640 0 client.16983 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 --2- 172.21.15.129:0/575833135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5615c0ef29a0 0x5615c0f01680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 >> 172.21.15.129:0/575833135 conn(0x5615c0eee010 msgr2=0x5615c0eef6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 shutdown_connections 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.712+0000 7f753624a740 1 -- 172.21.15.129:0/575833135 wait complete. 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f753624a740 20 client.16983 trim_cache size 0 max 16384 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:31:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:31:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 msgr2=0x7f75180a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 0x7f75180a50f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f752409b730 tx=0x7f75240963e0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 msgr2=0x7f752c1a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f752400cad0 tx=0x7f752400cf90 comp rx=0 tx=0).stop 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 shutdown_connections 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f75180a2c30 0x7f75180a50f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f752c101530 0x7f752c1a1510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f752c100b70 0x7f752c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 --2- 172.21.15.129:0/2128233006 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f752c0ffe10 0x7f752c199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 >> 172.21.15.129:0/2128233006 conn(0x7f752c0fb970 msgr2=0x7f752c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 shutdown_connections 2024-09-05T01:31:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:01.713+0000 7f75333d6640 1 -- 172.21.15.129:0/2128233006 wait complete. 2024-09-05T01:31:01.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:01 smithi129 ceph-mon[30979]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 758 op/s 2024-09-05T01:31:01.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:01 smithi194 ceph-mon[34941]: pgmap v824: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 7.6 MiB/s wr, 758 op/s 2024-09-05T01:31:02.778 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:31:03.190 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:31:03.616 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:03 smithi129 ceph-mon[30979]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-09-05T01:31:03.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:03 smithi183 ceph-mon[34589]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 -- 172.21.15.129:0/2895093866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc101450 msgr2=0x7f45cc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/2895093866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc101450 0x7f45cc105920 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009900 tx=0x7f45b402a470 comp rx=0 tx=0).stop 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 -- 172.21.15.129:0/2895093866 shutdown_connections 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/2895093866 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc105f40 0x7f45cc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/2895093866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc101450 0x7f45cc105920 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/2895093866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 0x7f45cc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 -- 172.21.15.129:0/2895093866 >> 172.21.15.129:0/2895093866 conn(0x7f45cc0fc7a0 msgr2=0x7f45cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 -- 172.21.15.129:0/2895093866 shutdown_connections 2024-09-05T01:31:03.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.694+0000 7f45d0ac3640 1 -- 172.21.15.129:0/2895093866 wait complete. 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.695+0000 7f45d0ac3640 1 Processor -- start 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.695+0000 7f45d0ac3640 1 -- start start 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 0x7f45cc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc101450 0x7f45cc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc118490 con 0x7f45cc105f40 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc118310 con 0x7f45cc100b30 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc118610 con 0x7f45cc101450 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56442/0 (socket says 172.21.15.129:56442) 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 -- 172.21.15.129:0/149710791 learned_addr learned my addr 172.21.15.129:0/149710791 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:03.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45ca59c640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 0x7f45cc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45c1d9b640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc101450 0x7f45cc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 -- 172.21.15.129:0/149710791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc101450 msgr2=0x7f45cc1a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc101450 0x7f45cc1a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 -- 172.21.15.129:0/149710791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 msgr2=0x7f45cc1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 0x7f45cc1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 -- 172.21.15.129:0/149710791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45cc19efd0 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45cad9d640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f45c400eaa0 tx=0x7f45c400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4004d20 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.696+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45cc19f2c0 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.697+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f45c4005020 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.697+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c40116d0 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.697+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45cc1ab5b0 con 0x7f45cc105f40 2024-09-05T01:31:03.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.697+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45cc105920 con 0x7f45cc105f40 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.699+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f45c400caf0 con 0x7f45cc105f40 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.699+0000 7f45c3fff640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 0x7f45b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.699+0000 7f45ca59c640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 0x7f45b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.699+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f45c40cf990 con 0x7f45cc105f40 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.700+0000 7f45ca59c640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 0x7f45b80a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19f9e0 tx=0x7f45bc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:03.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.703+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45c4016030 con 0x7f45cc105f40 2024-09-05T01:31:03.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:03 smithi194 ceph-mon[34941]: pgmap v825: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 8.1 MiB/s wr, 748 op/s 2024-09-05T01:31:03.916 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.914+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45cc1ab840 con 0x7f45cc105f40 2024-09-05T01:31:03.917 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.915+0000 7f45c3fff640 1 -- 172.21.15.129:0/149710791 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f45c409c4e0 con 0x7f45cc105f40 2024-09-05T01:31:03.917 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:31:03.917 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:31:03.917 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:31:03.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 msgr2=0x7f45b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:03.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 0x7f45b80a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f45cc19f9e0 tx=0x7f45bc009110 comp rx=0 tx=0).stop 2024-09-05T01:31:03.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 msgr2=0x7f45cc19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:03.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f45c400eaa0 tx=0x7f45c400eea0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.919 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 shutdown_connections 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f45b80a2b80 0x7f45b80a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45cc105f40 0x7f45cc19e8a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f45cc101450 0x7f45cc1a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 --2- 172.21.15.129:0/149710791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45cc100b30 0x7f45cc1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 >> 172.21.15.129:0/149710791 conn(0x7f45cc0fc7a0 msgr2=0x7f45cc10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 shutdown_connections 2024-09-05T01:31:03.920 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:03.917+0000 7f45d0ac3640 1 -- 172.21.15.129:0/149710791 wait complete. 2024-09-05T01:31:04.572 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:04 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/149710791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:04.600 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 43362322 --release --await' 2024-09-05T01:31:04.600 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 43362322 --release --await 2024-09-05T01:31:04.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:04 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/149710791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:04.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:04 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/149710791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:04.847 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:31:05.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 840.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:31:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 -- 172.21.15.129:0/1195499393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584100a60 msgr2=0x7f2584100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.359 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 --2- 172.21.15.129:0/1195499393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584100a60 0x7f2584100ec0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f256c009930 tx=0x7f256c02a420 comp rx=0 tx=0).stop 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 -- 172.21.15.129:0/1195499393 shutdown_connections 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 --2- 172.21.15.129:0/1195499393 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f258410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 --2- 172.21.15.129:0/1195499393 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584100a60 0x7f2584100ec0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 --2- 172.21.15.129:0/1195499393 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584106a60 0x7f2584106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 -- 172.21.15.129:0/1195499393 >> 172.21.15.129:0/1195499393 conn(0x7f25840fc780 msgr2=0x7f25840feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 -- 172.21.15.129:0/1195499393 shutdown_connections 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.354+0000 7f2589509640 1 -- 172.21.15.129:0/1195499393 wait complete. 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 Processor -- start 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 -- start start 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 0x7f2584197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f2584197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2584110d10 con 0x7f2584106a60 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2584110b90 con 0x7f2584101400 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2589509640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2584110e90 con 0x7f2584100a60 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56470/0 (socket says 172.21.15.129:56470) 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 -- 172.21.15.129:0/496678088 learned_addr learned my addr 172.21.15.129:0/496678088 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2582ffd640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 0x7f2584197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f257bfff640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f2584197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 -- 172.21.15.129:0/496678088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 msgr2=0x7f2584197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 0x7f2584197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 -- 172.21.15.129:0/496678088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 msgr2=0x7f2584197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f2584197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f25837fe640 1 -- 172.21.15.129:0/496678088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258419c2b0 con 0x7f2584106a60 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f2582ffd640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 0x7f2584197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.355+0000 7f257bfff640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f2584197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f25837fe640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cad0 tx=0x7f257c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f2580ff9640 1 -- 172.21.15.129:0/496678088 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00ae20 con 0x7f2584106a60 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f2580ff9640 1 -- 172.21.15.129:0/496678088 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f257c005020 con 0x7f2584106a60 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f2580ff9640 1 -- 172.21.15.129:0/496678088 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f6b0 con 0x7f2584106a60 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258419c5a0 con 0x7f2584106a60 2024-09-05T01:31:05.362 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.356+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a3e80 con 0x7f2584106a60 2024-09-05T01:31:05.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.358+0000 7f2580ff9640 1 -- 172.21.15.129:0/496678088 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f257c0051d0 con 0x7f2584106a60 2024-09-05T01:31:05.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.358+0000 7f258b8e2740 1 Processor -- start 2024-09-05T01:31:05.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.358+0000 7f258b8e2740 1 -- start start 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543beda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563543b4d620 con 0x563543bf1e00 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563543af4e50 con 0x563543bed600 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f258b8e2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563543b495e0 con 0x563543bedfa0 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56476/0 (socket says 172.21.15.129:56476) 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 -- 172.21.15.129:0/4226734080 learned_addr learned my addr 172.21.15.129:0/4226734080 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f2580ff9640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 0x7f25700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.368 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f2582ffd640 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543beda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f25837fe640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 0x7f25700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f2580ff9640 1 -- 172.21.15.129:0/496678088 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f257c0cf5a0 con 0x7f2584106a60 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 -- 172.21.15.129:0/4226734080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 msgr2=0x563543bf9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 -- 172.21.15.129:0/4226734080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 msgr2=0x563543beda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543beda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.359+0000 7f257bfff640 1 -- 172.21.15.129:0/4226734080 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563543ae5d40 con 0x563543bf1e00 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.360+0000 7f257bfff640 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f256c005ec0 tx=0x7f256c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a1bc409218d2a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.360+0000 7f25837fe640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 0x7f25700a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f257c061920 tx=0x7f257c0665c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.360+0000 7f25797fa640 1 -- 172.21.15.129:0/4226734080 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c035050 con 0x563543bf1e00 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.360+0000 7f25797fa640 1 -- 172.21.15.129:0/4226734080 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f256c0044d0 con 0x563543bf1e00 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.360+0000 7f25797fa640 1 -- 172.21.15.129:0/4226734080 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c0046c0 con 0x563543bf1e00 2024-09-05T01:31:05.369 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- 172.21.15.129:0/4226734080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 msgr2=0x563543bf21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f256c005ec0 tx=0x7f256c005b50 comp rx=0 tx=0).stop 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- 172.21.15.129:0/4226734080 shutdown_connections 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543beda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- 172.21.15.129:0/4226734080 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543bf21e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- 172.21.15.129:0/4226734080 >> 172.21.15.129:0/4226734080 conn(0x563543be40e0 msgr2=0x563543be44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- 172.21.15.129:0/4226734080 shutdown_connections 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- 172.21.15.129:0/4226734080 wait complete. 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- start start 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 Processor -- start 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bed600 0x563543bf7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.370 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 0x563543bf7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563543bfa1c0 con 0x563543bedfa0 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563543ad5f10 con 0x563543bf1e00 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563543bfa340 con 0x563543bed600 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f258b8e2740 10 client.0 _ll_register_callbacks cb 0x563543af9db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.361+0000 7f2582ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56482/0 (socket says 172.21.15.129:56482) 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 -- 172.21.15.129:0/2895014107 learned_addr learned my addr 172.21.15.129:0/2895014107 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f257bfff640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bed600 0x563543bf7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f257bfff640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 0x563543bf7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 -- 172.21.15.129:0/2895014107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bed600 msgr2=0x563543bf7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.371 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bed600 0x563543bf7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 -- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 msgr2=0x563543bf7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 0x563543bf7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 -- 172.21.15.129:0/2895014107 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354385c580 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f257bfff640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 0x563543bf7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2574013710 tx=0x7f2574013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c146db85a0d41bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f256a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574011dd0 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2574004020 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f256c0044d0 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2574004220 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f258b8e2740 10 client.17001 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.362+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563543926960 con 0x563543bedfa0 2024-09-05T01:31:05.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f256c0044d0 con 0x563543bedfa0 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f257401a070 con 0x563543bedfa0 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 10 client.17001 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 10 client.17001 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 20 client.17001 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c07300 0x563543c09720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 4 client.17001 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563543b5ab90 con 0x563543c07300 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.363+0000 7f257bfff640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c07300 0x563543c09720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f256a7fc640 10 client.17001 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f257bfff640 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c07300 0x563543c09720 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ea77107ab63234 server_cookie=a111dd61b3759517 in_seq=0 out_seq=0 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f256a7fc640 1 -- 172.21.15.129:0/2895014107 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563543b5ab90 con 0x563543c07300 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f256a7fc640 10 client.17001 handle_command_reply: tid=0 2024-09-05T01:31:05.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 client.17001 shutdown 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 msgr2=0x563543bf76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2574013710 tx=0x7f2574013bd0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 shutdown_connections 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c07300 0x563543c09720 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bf1e00 0x563543bf7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bf76a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f256a7fc640 0 client.17001 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2895014107 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bed600 0x563543bf7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.364+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 >> 172.21.15.129:0/2895014107 conn(0x563543be40e0 msgr2=0x563543bf3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.365+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 shutdown_connections 2024-09-05T01:31:05.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.365+0000 7f258b8e2740 1 -- 172.21.15.129:0/2895014107 wait complete. 2024-09-05T01:31:05.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.365+0000 7f258b8e2740 20 client.17001 trim_cache size 0 max 16384 2024-09-05T01:31:05.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 Processor -- start 2024-09-05T01:31:05.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 -- start start 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25841af4f0 con 0x563543bf1e00 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25841acdc0 con 0x563543bed600 2024-09-05T01:31:05.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.381+0000 7f258b8e2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25841aac10 con 0x563543bedfa0 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56496/0 (socket says 172.21.15.129:56496) 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 -- 172.21.15.129:0/1599492146 learned_addr learned my addr 172.21.15.129:0/1599492146 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f2582ffd640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 -- 172.21.15.129:0/1599492146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 msgr2=0x563543bf6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 -- 172.21.15.129:0/1599492146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 msgr2=0x563543bef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 -- 172.21.15.129:0/1599492146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563543be3e20 con 0x563543bf1e00 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f2582ffd640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f257bfff640 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f256c0057c0 tx=0x7f256c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc409efcfc0d4104 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f2578ff9640 1 -- 172.21.15.129:0/1599492146 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c035070 con 0x563543bf1e00 2024-09-05T01:31:05.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f2578ff9640 1 -- 172.21.15.129:0/1599492146 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f256c031d30 con 0x563543bf1e00 2024-09-05T01:31:05.386 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.382+0000 7f2578ff9640 1 -- 172.21.15.129:0/1599492146 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c04dac0 con 0x563543bf1e00 2024-09-05T01:31:05.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- 172.21.15.129:0/1599492146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 msgr2=0x563543c0eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f256c0057c0 tx=0x7f256c005d50 comp rx=0 tx=0).stop 2024-09-05T01:31:05.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- 172.21.15.129:0/1599492146 shutdown_connections 2024-09-05T01:31:05.388 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bf1e00 0x563543c0eff0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bedfa0 0x563543bf6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 --2- 172.21.15.129:0/1599492146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- 172.21.15.129:0/1599492146 >> 172.21.15.129:0/1599492146 conn(0x563543be40e0 msgr2=0x563543bf5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- 172.21.15.129:0/1599492146 shutdown_connections 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- 172.21.15.129:0/1599492146 wait complete. 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 -- start start 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.383+0000 7f258b8e2740 1 Processor -- start 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bff820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 0x563543bfc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25841ab280 con 0x563543bedfa0 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f25841ac6f0 con 0x563543bed600 2024-09-05T01:31:05.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f25841ab950 con 0x563543bf1e00 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 10 client.0 _ll_register_callbacks cb 0x7f2574009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:56510/0 (socket says 172.21.15.129:56510) 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 -- 172.21.15.129:0/2131753963 learned_addr learned my addr 172.21.15.129:0/2131753963 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f257bfff640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f257bfff640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 0x563543bfc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 -- 172.21.15.129:0/2131753963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 msgr2=0x563543bfc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 0x563543bfc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 -- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 msgr2=0x563543bff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 -- 172.21.15.129:0/2131753963 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563543bfcd90 con 0x563543bedfa0 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f257bfff640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 0x563543bfc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:05.390 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f2582ffd640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f2574011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69c321d56ceaf98 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f256a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257402d020 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f257401bb80 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.384+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f256c031d30 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2574038050 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 10 client.17007 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563543c72b30 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f256c031d30 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f257401a040 con 0x563543bedfa0 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 10 client.17007 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 10 client.17007 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 20 client.17007 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:05.391 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c732a0 0x563543c756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:05.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 4 client.17007 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "43362322", "release": true, "await": true}] 2024-09-05T01:31:05.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "43362322", "release": true, "await": true}) -- 0x563543b550a0 con 0x563543c732a0 2024-09-05T01:31:05.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.385+0000 7f257bfff640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c732a0 0x563543c756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:05.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.386+0000 7f256a7fc640 10 client.17007 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:05.392 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.386+0000 7f257bfff640 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c732a0 0x563543c756c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57006764db07f209 server_cookie=d627e41b51456e9c in_seq=0 out_seq=0 2024-09-05T01:31:05.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f256a7fc640 1 -- 172.21.15.129:0/2131753963 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563543b550a0 con 0x563543c732a0 2024-09-05T01:31:05.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f256a7fc640 10 client.17007 handle_command_reply: tid=0 2024-09-05T01:31:05.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 client.17007 shutdown 2024-09-05T01:31:05.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 msgr2=0x563543bffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.446 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2574009d90 tx=0x7f2574011880 comp rx=0 tx=0).stop 2024-09-05T01:31:05.447 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 shutdown_connections 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563543c732a0 0x563543c756c0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563543bf1e00 0x563543bfc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563543bedfa0 0x563543bffd60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 --2- 172.21.15.129:0/2131753963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563543bed600 0x563543bff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f256a7fc640 0 client.17007 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.444+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 >> 172.21.15.129:0/2131753963 conn(0x563543be40e0 msgr2=0x563543bf6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.446+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 shutdown_connections 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.446+0000 7f258b8e2740 1 -- 172.21.15.129:0/2131753963 wait complete. 2024-09-05T01:31:05.448 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.446+0000 7f258b8e2740 20 client.17007 trim_cache size 0 max 16384 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 60, 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "43362322": { 2024-09-05T01:31:05.449 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 60, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 131.6, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:31:05.450 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.447+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 msgr2=0x7f25700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.451 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.447+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 0x7f25700a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f257c061920 tx=0x7f257c0665c0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.447+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 msgr2=0x7f258419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.447+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cad0 tx=0x7f257c00cf90 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 shutdown_connections 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f25700a2b80 0x7f25700a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2584106a60 0x7f258419bbd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2584101400 0x7f2584197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 --2- 172.21.15.129:0/496678088 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2584100a60 0x7f2584197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 >> 172.21.15.129:0/496678088 conn(0x7f25840fc780 msgr2=0x7f25840fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 shutdown_connections 2024-09-05T01:31:05.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:05.448+0000 7f2589509640 1 -- 172.21.15.129:0/496678088 wait complete. 2024-09-05T01:31:05.562 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:05 smithi183 ceph-mon[34589]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-09-05T01:31:05.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:05 smithi129 ceph-mon[30979]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-09-05T01:31:05.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:05 smithi194 ceph-mon[34941]: pgmap v826: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.6 MiB/s wr, 613 op/s 2024-09-05T01:31:06.253 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '43362322', total seconds elapsed: 15.1 2024-09-05T01:31:07.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:07 smithi183 ceph-mon[34589]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2024-09-05T01:31:07.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:07 smithi129 ceph-mon[30979]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2024-09-05T01:31:07.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:07 smithi194 ceph-mon[34941]: pgmap v827: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2024-09-05T01:31:09.552 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:09 smithi194 ceph-mon[34941]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.2 MiB/s wr, 521 op/s 2024-09-05T01:31:09.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:09 smithi183 ceph-mon[34589]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.2 MiB/s wr, 521 op/s 2024-09-05T01:31:09.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:09 smithi129 ceph-mon[30979]: pgmap v828: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 7.2 MiB/s wr, 521 op/s 2024-09-05T01:31:10.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 845.0 s, 905.1 tps, lat 254.373 ms stddev 1018.259 2024-09-05T01:31:11.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:11 smithi183 ceph-mon[34589]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 418 op/s 2024-09-05T01:31:11.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:11 smithi129 ceph-mon[30979]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 418 op/s 2024-09-05T01:31:11.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:11 smithi194 ceph-mon[34941]: pgmap v829: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 418 op/s 2024-09-05T01:31:13.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:13 smithi183 ceph-mon[34589]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.5 MiB/s wr, 452 op/s 2024-09-05T01:31:13.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:13 smithi129 ceph-mon[30979]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.5 MiB/s wr, 452 op/s 2024-09-05T01:31:13.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:13 smithi194 ceph-mon[34941]: pgmap v830: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 6.5 MiB/s wr, 452 op/s 2024-09-05T01:31:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.855+0000 7f71e678d640 1 -- 172.21.15.129:0/3946086922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0100a40 msgr2=0x7f71e0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.855+0000 7f71e678d640 1 --2- 172.21.15.129:0/3946086922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0100a40 0x7f71e0100e40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009860 tx=0x7f71d002a340 comp rx=0 tx=0).stop 2024-09-05T01:31:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 -- 172.21.15.129:0/3946086922 shutdown_connections 2024-09-05T01:31:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 --2- 172.21.15.129:0/3946086922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e01025f0 0x7f71e0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.857 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 --2- 172.21.15.129:0/3946086922 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e0101c30 0x7f71e01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 --2- 172.21.15.129:0/3946086922 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0100a40 0x7f71e0100e40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 -- 172.21.15.129:0/3946086922 >> 172.21.15.129:0/3946086922 conn(0x7f71e00fc1e0 msgr2=0x7f71e00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 -- 172.21.15.129:0/3946086922 shutdown_connections 2024-09-05T01:31:14.858 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.856+0000 7f71e678d640 1 -- 172.21.15.129:0/3946086922 wait complete. 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.857+0000 7f71e678d640 1 Processor -- start 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.857+0000 7f71e678d640 1 -- start start 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 0x7f71e0199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 0x7f71e019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71dffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 0x7f71e0199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71dffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 0x7f71e0199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55684/0 (socket says 172.21.15.129:55684) 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 0x7f71e019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39736/0 (socket says 172.21.15.129:39736) 2024-09-05T01:31:14.859 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 0x7f71e019a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:48740/0 (socket says 172.21.15.129:48740) 2024-09-05T01:31:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71dffff640 1 -- 172.21.15.129:0/3379677601 learned_addr learned my addr 172.21.15.129:0/3379677601 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010c550 con 0x7f71e0100a40 2024-09-05T01:31:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010c3d0 con 0x7f71e0101c30 2024-09-05T01:31:14.860 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.858+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71e010c6d0 con 0x7f71e01025f0 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3379677601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 msgr2=0x7f71e019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 0x7f71e019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 msgr2=0x7f71e0199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 0x7f71e0199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e01a1ac0 con 0x7f71e01025f0 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.859+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f71d800ed70 tx=0x7f71d800c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.860+0000 7f71dd7fa640 1 -- 172.21.15.129:0/3379677601 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8003970 con 0x7f71e01025f0 2024-09-05T01:31:14.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.860+0000 7f71dd7fa640 1 -- 172.21.15.129:0/3379677601 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d8004020 con 0x7f71e01025f0 2024-09-05T01:31:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.860+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e01a1db0 con 0x7f71e01025f0 2024-09-05T01:31:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.861+0000 7f71dd7fa640 1 -- 172.21.15.129:0/3379677601 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8017770 con 0x7f71e01025f0 2024-09-05T01:31:14.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.861+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e01a21c0 con 0x7f71e01025f0 2024-09-05T01:31:14.867 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.864+0000 7f71e9607740 1 Processor -- start 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 -- start start 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a4ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 0x562af7a52db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a53380 0x562af7a55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562af79a02f0 con 0x562af7a53380 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562af7948270 con 0x562af7a4d9b0 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71e9607740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562af799c2b0 con 0x562af7a4e2f0 2024-09-05T01:31:14.868 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71dd7fa640 1 -- 172.21.15.129:0/3379677601 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f71d8003b10 con 0x7f71e01025f0 2024-09-05T01:31:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a4ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.865+0000 7f71df7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a4ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:48746/0 (socket says 172.21.15.129:48746) 2024-09-05T01:31:14.869 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.866+0000 7f71df7fe640 1 -- 172.21.15.129:0/2598840448 learned_addr learned my addr 172.21.15.129:0/2598840448 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.866+0000 7f71dffff640 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 0x562af7a52db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.867+0000 7f71dffff640 1 -- 172.21.15.129:0/2598840448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 msgr2=0x562af7a4ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.867+0000 7f71dffff640 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a4ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.867+0000 7f71dffff640 1 -- 172.21.15.129:0/2598840448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a53380 msgr2=0x562af7a55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.867+0000 7f71dffff640 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a53380 0x562af7a55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.867+0000 7f71dffff640 1 -- 172.21.15.129:0/2598840448 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af7947340 con 0x562af7a4e2f0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71dd7fa640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 0x7f71c80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71dffff640 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 0x562af7a52db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f71d002a930 tx=0x7f71d0030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281e265386547b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71df7fe640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 0x7f71c80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71dd7fa640 1 -- 172.21.15.129:0/3379677601 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71d80cf540 con 0x7f71e01025f0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71c67fc640 1 -- 172.21.15.129:0/2598840448 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0043020 con 0x562af7a4e2f0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71c67fc640 1 -- 172.21.15.129:0/2598840448 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d0038040 con 0x562af7a4e2f0 2024-09-05T01:31:14.870 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.868+0000 7f71c67fc640 1 -- 172.21.15.129:0/2598840448 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037040 con 0x562af7a4e2f0 2024-09-05T01:31:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 -- 172.21.15.129:0/2598840448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 msgr2=0x562af7a52db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 0x562af7a52db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f71d002a930 tx=0x7f71d0030bd0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71df7fe640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 0x7f71c80a52e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f71d400a5c0 tx=0x7f71d400a810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 -- 172.21.15.129:0/2598840448 shutdown_connections 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a53380 0x562af7a55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a4e2f0 0x562af7a52db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 --2- 172.21.15.129:0/2598840448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a4ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.869+0000 7f71e9607740 1 -- 172.21.15.129:0/2598840448 >> 172.21.15.129:0/2598840448 conn(0x562af7a38010 msgr2=0x562af7a38410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.870+0000 7f71e9607740 1 -- 172.21.15.129:0/2598840448 shutdown_connections 2024-09-05T01:31:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.872+0000 7f71e9607740 1 -- 172.21.15.129:0/2598840448 wait complete. 2024-09-05T01:31:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.872+0000 7f71e9607740 1 -- start start 2024-09-05T01:31:14.873 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.872+0000 7f71e9607740 1 Processor -- start 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a46f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a4bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562af7a3c870 con 0x562af7a4d9b0 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562af7929290 con 0x562af7a4e2f0 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x562af7a3c9f0 con 0x562af7a53380 2024-09-05T01:31:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.874+0000 7f71e9607740 10 client.0 _ll_register_callbacks cb 0x562af795a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71dffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71dffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55698/0 (socket says 172.21.15.129:55698) 2024-09-05T01:31:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71dffff640 1 -- 172.21.15.129:0/3673151945 learned_addr learned my addr 172.21.15.129:0/3673151945 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a4bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71df7fe640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a46f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 msgr2=0x562af7a46f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a46f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3673151945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 msgr2=0x562af7a469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 -- 172.21.15.129:0/3673151945 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af76b3210 con 0x562af7a53380 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a4bf40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71d800cbe0 tx=0x7f71d809b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa65489c15718d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.875+0000 7f71bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80a45a0 con 0x562af7a53380 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71d8004540 con 0x562af7a53380 2024-09-05T01:31:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d0038040 con 0x562af7a53380 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71d80cf170 con 0x562af7a53380 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71e9607740 10 client.26443 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.876+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562af777f620 con 0x562af7a53380 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d0038040 con 0x562af7a53380 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f71d800b030 con 0x562af7a53380 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 10 client.26443 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:14.878 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 10 client.26443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 20 client.26443 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7a40cd0 0x562af7a43140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 4 client.26443 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.877+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562af79ada20 con 0x562af7a40cd0 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.878+0000 7f71dffff640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7a40cd0 0x562af7a43140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.878+0000 7f71dffff640 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7a40cd0 0x562af7a43140 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30f5a560daa3268f server_cookie=2a5f37eaf92626f in_seq=0 out_seq=0 2024-09-05T01:31:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.879+0000 7f71bf7fe640 10 client.26443 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.879+0000 7f71bf7fe640 1 -- 172.21.15.129:0/3673151945 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562af79ada20 con 0x562af7a40cd0 2024-09-05T01:31:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.879+0000 7f71bf7fe640 10 client.26443 handle_command_reply: tid=0 2024-09-05T01:31:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.879+0000 7f71e9607740 1 client.26443 shutdown 2024-09-05T01:31:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.880+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 msgr2=0x562af7a4bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.880+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a4bf40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f71d800cbe0 tx=0x7f71d809b580 comp rx=0 tx=0).stop 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 shutdown_connections 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7a40cd0 0x562af7a43140 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a4bf40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a46f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 --2- 172.21.15.129:0/3673151945 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 >> 172.21.15.129:0/3673151945 conn(0x562af7a38010 msgr2=0x562af7a39820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:14.882 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.881+0000 7f71bf7fe640 0 client.26443 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.883+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 shutdown_connections 2024-09-05T01:31:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.884+0000 7f71e9607740 1 -- 172.21.15.129:0/3673151945 wait complete. 2024-09-05T01:31:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.884+0000 7f71e9607740 20 client.26443 trim_cache size 0 max 16384 2024-09-05T01:31:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 Processor -- start 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 -- start start 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a4dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e01a2780 con 0x562af7a4d9b0 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0071fd0 con 0x562af7a4e2f0 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.916+0000 7f71e9607740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71e006fe20 con 0x562af7a53380 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71dffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a4dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39754/0 (socket says 172.21.15.129:39754) 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1301851840 learned_addr learned my addr 172.21.15.129:0/1301851840 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71df7fe640 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1301851840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 msgr2=0x562af7a459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.918 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1301851840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 msgr2=0x562af7a4dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a4dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1301851840 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562af7a52890 con 0x562af7a53380 2024-09-05T01:31:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71dffff640 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a4dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f71d809b630 tx=0x7f71d800cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80844d9eda34592 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71c5ffb640 1 -- 172.21.15.129:0/1301851840 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80979f0 con 0x562af7a53380 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.917+0000 7f71c5ffb640 1 -- 172.21.15.129:0/1301851840 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d80afe50 con 0x562af7a53380 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71c5ffb640 1 -- 172.21.15.129:0/1301851840 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8004540 con 0x562af7a53380 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 -- 172.21.15.129:0/1301851840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 msgr2=0x562af7a45f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f71d809b630 tx=0x7f71d800cd10 comp rx=0 tx=0).stop 2024-09-05T01:31:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 -- 172.21.15.129:0/1301851840 shutdown_connections 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a45f30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4e2f0 0x562af7a459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 --2- 172.21.15.129:0/1301851840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4d9b0 0x562af7a4dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.918+0000 7f71e9607740 1 -- 172.21.15.129:0/1301851840 >> 172.21.15.129:0/1301851840 conn(0x562af7a38010 msgr2=0x562af7a469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.919+0000 7f71e9607740 1 -- 172.21.15.129:0/1301851840 shutdown_connections 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.919+0000 7f71e9607740 1 -- 172.21.15.129:0/1301851840 wait complete. 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.919+0000 7f71e9607740 1 -- start start 2024-09-05T01:31:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.919+0000 7f71e9607740 1 Processor -- start 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a77930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4e2f0 0x562af7a77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0070490 con 0x562af7a4e2f0 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0071900 con 0x562af7a4d9b0 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71e0070b60 con 0x562af7a53380 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e9607740 10 client.0 _ll_register_callbacks cb 0x7f71d8097bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:39758/0 (socket says 172.21.15.129:39758) 2024-09-05T01:31:14.923 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1030287319 learned_addr learned my addr 172.21.15.129:0/1030287319 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71df7fe640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4e2f0 0x562af7a77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 msgr2=0x562af7a77930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a77930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.920+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1030287319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4e2f0 msgr2=0x562af7a77e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4e2f0 0x562af7a77e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e4d2a640 1 -- 172.21.15.129:0/1030287319 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562af7a79490 con 0x562af7a53380 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e4d2a640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71d8097d90 tx=0x7f71d80a4420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6a838fba1cc97df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d80a9060 con 0x562af7a53380 2024-09-05T01:31:14.924 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71d80afe50 con 0x562af7a53380 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71d8004540 con 0x562af7a53380 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f71d80a45a0 con 0x562af7a53380 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e9607740 10 client.26449 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.921+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562af7a7a7a0 con 0x562af7a53380 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71d80afe50 con 0x562af7a53380 2024-09-05T01:31:14.925 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f71d809a7f0 con 0x562af7a53380 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 10 client.26449 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 20 client.26449 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7acf0f0 0x562af7ad14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 4 client.26449 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.922+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562af79a7d70 con 0x562af7acf0f0 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.923+0000 7f71dffff640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7acf0f0 0x562af7ad14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.924+0000 7f71dffff640 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7acf0f0 0x562af7ad14e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9638565e40a7c6d server_cookie=700dfe188d13b7c4 in_seq=0 out_seq=0 2024-09-05T01:31:14.926 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:14.925+0000 7f71bf7fe640 10 client.26449 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:15.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 850.0 s, 922.0 tps, lat 109.874 ms stddev 97.818 2024-09-05T01:31:15.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:15 smithi183 ceph-mon[34589]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2024-09-05T01:31:15.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:15 smithi129 ceph-mon[30979]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2024-09-05T01:31:15.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:15 smithi194 ceph-mon[34941]: pgmap v831: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 7.7 MiB/s wr, 503 op/s 2024-09-05T01:31:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71bf7fe640 1 -- 172.21.15.129:0/1030287319 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562af79a7d70 con 0x562af7acf0f0 2024-09-05T01:31:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71bf7fe640 10 client.26449 handle_command_reply: tid=0 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 client.26449 shutdown 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 msgr2=0x562af7a78e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f71d8097d90 tx=0x7f71d80a4420 comp rx=0 tx=0).stop 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 shutdown_connections 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x562af7acf0f0 0x562af7ad14e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x562af7a53380 0x562af7a78e80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562af7a4e2f0 0x562af7a77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 --2- 172.21.15.129:0/1030287319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562af7a4d9b0 0x562af7a77930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 >> 172.21.15.129:0/1030287319 conn(0x562af7a38010 msgr2=0x562af7a459f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71bf7fe640 0 client.26449 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.272+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 shutdown_connections 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.273+0000 7f71e9607740 1 -- 172.21.15.129:0/1030287319 wait complete. 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.273+0000 7f71e9607740 20 client.26449 trim_cache size 0 max 16384 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:31:16.275 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:31:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 msgr2=0x7f71c80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 0x7f71c80a52e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f71d400a5c0 tx=0x7f71d400a810 comp rx=0 tx=0).stop 2024-09-05T01:31:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 msgr2=0x7f71e01a1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f71d800ed70 tx=0x7f71d800c6e0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 shutdown_connections 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f71c80a2e20 0x7f71c80a52e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f71e01025f0 0x7f71e01a1450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71e0101c30 0x7f71e019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 --2- 172.21.15.129:0/3379677601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e0100a40 0x7f71e0199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 >> 172.21.15.129:0/3379677601 conn(0x7f71e00fc1e0 msgr2=0x7f71e00698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 shutdown_connections 2024-09-05T01:31:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:16.274+0000 7f71e678d640 1 -- 172.21.15.129:0/3379677601 wait complete. 2024-09-05T01:31:17.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:17 smithi183 ceph-mon[34589]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-09-05T01:31:17.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:17 smithi129 ceph-mon[30979]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-09-05T01:31:17.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:17 smithi194 ceph-mon[34941]: pgmap v832: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 9.4 MiB/s wr, 582 op/s 2024-09-05T01:31:19.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:19 smithi183 ceph-mon[34589]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 790 op/s 2024-09-05T01:31:19.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:19 smithi129 ceph-mon[30979]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 790 op/s 2024-09-05T01:31:19.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:19 smithi194 ceph-mon[34941]: pgmap v833: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 13 MiB/s wr, 790 op/s 2024-09-05T01:31:20.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 855.0 s, 954.6 tps, lat 104.686 ms stddev 80.470 2024-09-05T01:31:21.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:21 smithi183 ceph-mon[34589]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-05T01:31:21.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:21 smithi129 ceph-mon[30979]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-05T01:31:21.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:21 smithi194 ceph-mon[34941]: pgmap v834: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-05T01:31:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:23 smithi183 ceph-mon[34589]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:23 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:31:23.745 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:23 smithi129 ceph-mon[30979]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:23.745 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:23 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:31:23.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:23 smithi194 ceph-mon[34941]: pgmap v835: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:23.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:23 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:31:24.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:24 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:31:24.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:24 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:31:24.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:24 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:31:24.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:24 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:31:24.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:24 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:31:24.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:24 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:31:24.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:24 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:31:24.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:24 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:31:24.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:24 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:31:25.207 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 860.0 s, 1013.2 tps, lat 99.341 ms stddev 59.481 2024-09-05T01:31:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:25 smithi129 ceph-mon[30979]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:26.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:25 smithi194 ceph-mon[34941]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:25 smithi183 ceph-mon[34589]: pgmap v836: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 15 MiB/s wr, 897 op/s 2024-09-05T01:31:27.797 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T01:31:27.797 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T01:31:27.797 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:31:28.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:27 smithi129 ceph-mon[30979]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 869 op/s 2024-09-05T01:31:28.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:27 smithi194 ceph-mon[34941]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 869 op/s 2024-09-05T01:31:28.283 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:31:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:27 smithi183 ceph-mon[34589]: pgmap v837: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 14 MiB/s wr, 869 op/s 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.006+0000 7f3777375640 1 -- 172.21.15.129:0/501797789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770102790 msgr2=0x7f37701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.006+0000 7f3777375640 1 --2- 172.21.15.129:0/501797789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770102790 0x7f37701067e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3768009900 tx=0x7f376802a3f0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 -- 172.21.15.129:0/501797789 shutdown_connections 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 --2- 172.21.15.129:0/501797789 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770106f10 0x7f377010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 --2- 172.21.15.129:0/501797789 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770102790 0x7f37701067e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 --2- 172.21.15.129:0/501797789 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770101de0 0x7f37701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 -- 172.21.15.129:0/501797789 >> 172.21.15.129:0/501797789 conn(0x7f37700fd8e0 msgr2=0x7f37700ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 -- 172.21.15.129:0/501797789 shutdown_connections 2024-09-05T01:31:29.010 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 -- 172.21.15.129:0/501797789 wait complete. 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.007+0000 7f3777375640 1 Processor -- start 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 -- start start 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770101de0 0x7f3770110c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770102790 0x7f3770111180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3770075720 con 0x7f3770101de0 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37700755a0 con 0x7f3770106f10 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3777375640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f37700758a0 con 0x7f3770102790 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3775912640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3775912640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55542/0 (socket says 172.21.15.129:55542) 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.008+0000 7f3775912640 1 -- 172.21.15.129:0/2649248371 learned_addr learned my addr 172.21.15.129:0/2649248371 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.011 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775111640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770101de0 0x7f3770110c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3774910640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770102790 0x7f3770111180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 -- 172.21.15.129:0/2649248371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770102790 msgr2=0x7f3770111180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770102790 0x7f3770111180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 -- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770101de0 msgr2=0x7f3770110c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770101de0 0x7f3770110c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.012 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 -- 172.21.15.129:0/2649248371 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37701b34b0 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3775912640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f376400d780 tx=0x7f376400dc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764017070 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f376400de90 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37701b3740 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.009+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764004c30 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.010+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701b3c50 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.011+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37701032c0 con 0x7f3770106f10 2024-09-05T01:31:29.013 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.012+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3764004dd0 con 0x7f3770106f10 2024-09-05T01:31:29.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.013+0000 7f376e7fc640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 0x7f37580a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.014+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f37640cedb0 con 0x7f3770106f10 2024-09-05T01:31:29.015 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.014+0000 7f3775111640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 0x7f37580a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.016 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.015+0000 7f3775111640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 0x7f37580a5180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37600099a0 tx=0x7f3760009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.018 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.018+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f376409b8a0 con 0x7f3770106f10 2024-09-05T01:31:29.399 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.398+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37701b3f40 con 0x7f3770106f10 2024-09-05T01:31:29.402 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:31:29.402 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:31:29.403 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.400+0000 7f376e7fc640 1 -- 172.21.15.129:0/2649248371 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f376409b1a0 con 0x7f3770106f10 2024-09-05T01:31:29.403 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:31:29.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.405+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 msgr2=0x7f37580a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.405+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 0x7f37580a5180 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37600099a0 tx=0x7f3760009500 comp rx=0 tx=0).stop 2024-09-05T01:31:29.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.405+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 msgr2=0x7f37701116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.405+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f376400d780 tx=0x7f376400dc40 comp rx=0 tx=0).stop 2024-09-05T01:31:29.406 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 shutdown_connections 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f37580a2cc0 0x7f37580a5180 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3770106f10 0x7f37701116c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3770102790 0x7f3770111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 --2- 172.21.15.129:0/2649248371 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3770101de0 0x7f3770110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 >> 172.21.15.129:0/2649248371 conn(0x7f37700fd8e0 msgr2=0x7f37700fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 shutdown_connections 2024-09-05T01:31:29.407 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:29.406+0000 7f3777375640 1 -- 172.21.15.129:0/2649248371 wait complete. 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 -- 172.21.15.129:0/3789885446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 msgr2=0x7fb460073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 --2- 172.21.15.129:0/3789885446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb460073180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb450009860 tx=0x7fb45002a340 comp rx=0 tx=0).stop 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 -- 172.21.15.129:0/3789885446 shutdown_connections 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 --2- 172.21.15.129:0/3789885446 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4600736c0 0x7fb460119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 --2- 172.21.15.129:0/3789885446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb460073180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 --2- 172.21.15.129:0/3789885446 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460109700 0x7fb460109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.441+0000 7fb467a04640 1 -- 172.21.15.129:0/3789885446 >> 172.21.15.129:0/3789885446 conn(0x7fb46006ece0 msgr2=0x7fb460071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.444+0000 7fb467a04640 1 -- 172.21.15.129:0/3789885446 shutdown_connections 2024-09-05T01:31:29.448 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.447+0000 7fb467a04640 1 -- 172.21.15.129:0/3789885446 wait complete. 2024-09-05T01:31:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.448+0000 7fb467a04640 1 Processor -- start 2024-09-05T01:31:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 -- start start 2024-09-05T01:31:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4600736c0 0x7fb46019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460109700 0x7fb4601a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb460113430 con 0x7fb4600736c0 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601132b0 con 0x7fb460072d20 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb467a04640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601135b0 con 0x7fb460109700 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55568/0 (socket says 172.21.15.129:55568) 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.449+0000 7fb4657a0640 1 -- 172.21.15.129:0/2663483672 learned_addr learned my addr 172.21.15.129:0/2663483672 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.450+0000 7fb4657a0640 1 -- 172.21.15.129:0/2663483672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460109700 msgr2=0x7fb4601a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.450+0000 7fb4657a0640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460109700 0x7fb4601a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.450+0000 7fb4657a0640 1 -- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4600736c0 msgr2=0x7fb46019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.450+0000 7fb4657a0640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4600736c0 0x7fb46019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.450+0000 7fb4657a0640 1 -- 172.21.15.129:0/2663483672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4601a5f20 con 0x7fb460072d20 2024-09-05T01:31:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.451+0000 7fb4657a0640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb45c01fa60 tx=0x7fb45c01ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.451+0000 7fb4567fc640 1 -- 172.21.15.129:0/2663483672 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c01dc50 con 0x7fb460072d20 2024-09-05T01:31:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.451+0000 7fb4567fc640 1 -- 172.21.15.129:0/2663483672 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45c01ddf0 con 0x7fb460072d20 2024-09-05T01:31:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.453+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4601a6210 con 0x7fb460072d20 2024-09-05T01:31:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.453+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4601a6760 con 0x7fb460072d20 2024-09-05T01:31:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.453+0000 7fb4567fc640 1 -- 172.21.15.129:0/2663483672 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c021440 con 0x7fb460072d20 2024-09-05T01:31:29.457 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.457+0000 7fb46a879740 1 Processor -- start 2024-09-05T01:31:29.459 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.457+0000 7fb46a879740 1 -- start start 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f869a0 0x556f75f86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f872e0 0x556f75f8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f75eea2f0 con 0x556f75f869a0 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f75e92270 con 0x556f75f872e0 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb46a879740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556f75ee62b0 con 0x556f75f8c370 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb464f9f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb464f9f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36412/0 (socket says 172.21.15.129:36412) 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb464f9f640 1 -- 172.21.15.129:0/2369481065 learned_addr learned my addr 172.21.15.129:0/2369481065 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.458+0000 7fb465fa1640 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f869a0 0x556f75f86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.460 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb464f9f640 1 -- 172.21.15.129:0/2369481065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f872e0 msgr2=0x556f75f8bda0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T01:31:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb464f9f640 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f872e0 0x556f75f8bda0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb464f9f640 1 -- 172.21.15.129:0/2369481065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f869a0 msgr2=0x556f75f86da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb464f9f640 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f869a0 0x556f75f86da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb464f9f640 1 -- 172.21.15.129:0/2369481065 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f75e91340 con 0x556f75f8c370 2024-09-05T01:31:29.461 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.459+0000 7fb4567fc640 1 -- 172.21.15.129:0/2663483672 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb45c005000 con 0x7fb460072d20 2024-09-05T01:31:29.462 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.460+0000 7fb4567fc640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 0x7fb44c0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.460+0000 7fb4567fc640 1 -- 172.21.15.129:0/2663483672 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb45c0e07e0 con 0x7fb460072d20 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.460+0000 7fb464f9f640 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4500057b0 tx=0x7fb450009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74798c4b5deef00b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.460+0000 7fb465fa1640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 0x7fb44c0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.461+0000 7fb44b7fe640 1 -- 172.21.15.129:0/2369481065 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450037040 con 0x556f75f8c370 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.461+0000 7fb44b7fe640 1 -- 172.21.15.129:0/2369481065 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb450040070 con 0x556f75f8c370 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.461+0000 7fb44b7fe640 1 -- 172.21.15.129:0/2369481065 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450003850 con 0x556f75f8c370 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 -- 172.21.15.129:0/2369481065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 msgr2=0x556f75f8e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4500057b0 tx=0x7fb450009490 comp rx=0 tx=0).stop 2024-09-05T01:31:29.463 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb465fa1640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 0x7fb44c0a5330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb458005fd0 tx=0x7fb458009b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 -- 172.21.15.129:0/2369481065 shutdown_connections 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f8e830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f872e0 0x556f75f8bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 --2- 172.21.15.129:0/2369481065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f869a0 0x556f75f86da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.462+0000 7fb46a879740 1 -- 172.21.15.129:0/2369481065 >> 172.21.15.129:0/2369481065 conn(0x556f75f82010 msgr2=0x556f75f82410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.463+0000 7fb46a879740 1 -- 172.21.15.129:0/2369481065 shutdown_connections 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.463+0000 7fb46a879740 1 -- 172.21.15.129:0/2369481065 wait complete. 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.463+0000 7fb46a879740 1 -- start start 2024-09-05T01:31:29.465 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.464+0000 7fb46a879740 1 Processor -- start 2024-09-05T01:31:29.466 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556f75f8f2e0 con 0x556f75f872e0 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556f75e73290 con 0x556f75f869a0 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.465+0000 7fb46a879740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556f75f8f460 con 0x556f75f8c370 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.466+0000 7fb46a879740 10 client.0 _ll_register_callbacks cb 0x556f75ea4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:29.467 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55570/0 (socket says 172.21.15.129:55570) 2024-09-05T01:31:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 -- 172.21.15.129:0/92652816 learned_addr learned my addr 172.21.15.129:0/92652816 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb464f9f640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 -- 172.21.15.129:0/92652816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 msgr2=0x556f75f98120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:31:29.468 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f98120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 -- 172.21.15.129:0/92652816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 msgr2=0x556f75f910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.467+0000 7fb4657a0640 1 -- 172.21.15.129:0/92652816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f75bfd210 con 0x556f75f869a0 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.468+0000 7fb4657a0640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0a95a0 tx=0x7fb45c0acc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b4529cf668f092 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.468+0000 7fb448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.468+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0046c0 con 0x556f75f869a0 2024-09-05T01:31:29.469 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.468+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb45c0a9240 con 0x556f75f869a0 2024-09-05T01:31:29.471 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.470+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb450040070 con 0x556f75f869a0 2024-09-05T01:31:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.471+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb45c0e0440 con 0x556f75f869a0 2024-09-05T01:31:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.471+0000 7fb46a879740 10 client.26603 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:29.472 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.471+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556f75cc9620 con 0x556f75f869a0 2024-09-05T01:31:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.471+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb450040070 con 0x556f75f869a0 2024-09-05T01:31:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.471+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb45c0b53b0 con 0x556f75f869a0 2024-09-05T01:31:29.473 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 10 client.26603 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 20 client.26603 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f75f9ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 4 client.26603 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f75ef7a20 con 0x556f75f9c860 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.472+0000 7fb464f9f640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f75f9ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.473+0000 7fb464f9f640 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f75f9ec50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc4dc6363b15ca5 server_cookie=7c8ed2c6172648e4 in_seq=0 out_seq=0 2024-09-05T01:31:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.473+0000 7fb448ff9640 10 client.26603 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.473+0000 7fb448ff9640 1 -- 172.21.15.129:0/92652816 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f75ef7a20 con 0x556f75f9c860 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.473+0000 7fb448ff9640 10 client.26603 handle_command_reply: tid=0 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.473+0000 7fb46a879740 1 client.26603 shutdown 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 msgr2=0x556f75f90ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0a95a0 tx=0x7fb45c0acc90 comp rx=0 tx=0).stop 2024-09-05T01:31:29.476 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 shutdown_connections 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f75f9ec50 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75f98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb448ff9640 0 client.26603 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 --2- 172.21.15.129:0/92652816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f90ba0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.474+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 >> 172.21.15.129:0/92652816 conn(0x556f75f82010 msgr2=0x556f75f83820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.475+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 shutdown_connections 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.475+0000 7fb46a879740 1 -- 172.21.15.129:0/92652816 wait complete. 2024-09-05T01:31:29.477 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.475+0000 7fb46a879740 20 client.26603 trim_cache size 0 max 16384 2024-09-05T01:31:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.503+0000 7fb46a879740 1 Processor -- start 2024-09-05T01:31:29.505 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 -- start start 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f980e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75fa3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b37a0 con 0x556f75f872e0 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b1070 con 0x556f75f869a0 2024-09-05T01:31:29.506 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb46a879740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601aeec0 con 0x556f75f8c370 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55572/0 (socket says 172.21.15.129:55572) 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb4657a0640 1 -- 172.21.15.129:0/1233296823 learned_addr learned my addr 172.21.15.129:0/1233296823 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.505+0000 7fb464f9f640 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 -- 172.21.15.129:0/1233296823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 msgr2=0x556f75fa3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75fa3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 -- 172.21.15.129:0/1233296823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 msgr2=0x556f75f980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 -- 172.21.15.129:0/1233296823 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f75f81e30 con 0x556f75f869a0 2024-09-05T01:31:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb4657a0640 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0cb450 tx=0x7fb45c0ab0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f0217df28d3bd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb44affd640 1 -- 172.21.15.129:0/1233296823 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0046c0 con 0x556f75f869a0 2024-09-05T01:31:29.508 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.506+0000 7fb44affd640 1 -- 172.21.15.129:0/1233296823 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45c0a9240 con 0x556f75f869a0 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.507+0000 7fb44affd640 1 -- 172.21.15.129:0/1233296823 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0ace30 con 0x556f75f869a0 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.507+0000 7fb46a879740 1 -- 172.21.15.129:0/1233296823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 msgr2=0x556f75fa3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.507+0000 7fb46a879740 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0cb450 tx=0x7fb45c0ab0e0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 -- 172.21.15.129:0/1233296823 shutdown_connections 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f8c370 0x556f75fa3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f872e0 0x556f75f980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 --2- 172.21.15.129:0/1233296823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75fa3550 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.509 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 -- 172.21.15.129:0/1233296823 >> 172.21.15.129:0/1233296823 conn(0x556f75f82010 msgr2=0x556f75f811c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 -- 172.21.15.129:0/1233296823 shutdown_connections 2024-09-05T01:31:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 -- 172.21.15.129:0/1233296823 wait complete. 2024-09-05T01:31:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 -- start start 2024-09-05T01:31:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.508+0000 7fb46a879740 1 Processor -- start 2024-09-05T01:31:29.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f872e0 0x556f75f95fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f8c370 0x556f75f96fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601af530 con 0x556f75f8c370 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601b09a0 con 0x556f75f869a0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb4601afc00 con 0x556f75f872e0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb46a879740 10 client.0 _ll_register_callbacks cb 0x7fb45c01ff90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb4657a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55574/0 (socket says 172.21.15.129:55574) 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb4657a0640 1 -- 172.21.15.129:0/3852188856 learned_addr learned my addr 172.21.15.129:0/3852188856 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.509+0000 7fb465fa1640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f8c370 0x556f75f96fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb464f9f640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f872e0 0x556f75f95fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 -- 172.21.15.129:0/3852188856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f872e0 msgr2=0x556f75f95fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f872e0 0x556f75f95fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 -- 172.21.15.129:0/3852188856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f8c370 msgr2=0x556f75f96fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f8c370 0x556f75f96fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 -- 172.21.15.129:0/3852188856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f75f975d0 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.510+0000 7fb4657a0640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b5310 tx=0x7fb45c0acc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53901638e306f3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.511+0000 7fb448ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.511+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0ac020 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.511+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb45c0ac1c0 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.511+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c0a9240 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.511+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb45c0a8920 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 10 client.26609 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556f760077f0 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb45c0a9240 con 0x556f75f869a0 2024-09-05T01:31:29.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb45c0aba90 con 0x556f75f869a0 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 10 client.26609 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 10 client.26609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 20 client.26609 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f76009fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 4 client.26609 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.512+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f75ef1d70 con 0x556f75f9c860 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.513+0000 7fb464f9f640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f76009fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.514+0000 7fb464f9f640 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f76009fa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4371dca4c5ce29 server_cookie=f67c92d57f29812f in_seq=0 out_seq=0 2024-09-05T01:31:29.517 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.514+0000 7fb448ff9640 10 client.26609 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.539+0000 7fb448ff9640 1 -- 172.21.15.129:0/3852188856 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f75ef1d70 con 0x556f75f9c860 2024-09-05T01:31:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.539+0000 7fb448ff9640 10 client.26609 handle_command_reply: tid=0 2024-09-05T01:31:29.540 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.539+0000 7fb46a879740 1 client.26609 shutdown 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 msgr2=0x556f75f95a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b5310 tx=0x7fb45c0acc90 comp rx=0 tx=0).stop 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 shutdown_connections 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556f75f9c860 0x556f76009fa0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556f75f8c370 0x556f75f96fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556f75f872e0 0x556f75f95fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 --2- 172.21.15.129:0/3852188856 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556f75f869a0 0x556f75f95a70 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0b5310 tx=0x7fb45c0acc90 comp rx=0 tx=0).stop 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 >> 172.21.15.129:0/3852188856 conn(0x556f75f82010 msgr2=0x556f75f82ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.541 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb448ff9640 0 client.26609 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.540+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 shutdown_connections 2024-09-05T01:31:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.541+0000 7fb46a879740 1 -- 172.21.15.129:0/3852188856 wait complete. 2024-09-05T01:31:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.541+0000 7fb46a879740 20 client.26609 trim_cache size 0 max 16384 2024-09-05T01:31:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:31:29.542 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:31:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:31:29.543 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.542+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 msgr2=0x7fb44c0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.542+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 0x7fb44c0a5330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb458005fd0 tx=0x7fb458009b40 comp rx=0 tx=0).stop 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.542+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 msgr2=0x7fb46019e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.542+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb45c01fa60 tx=0x7fb45c01ff20 comp rx=0 tx=0).stop 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 shutdown_connections 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb44c0a2e70 0x7fb44c0a5330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb460109700 0x7fb4601a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4600736c0 0x7fb46019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.544 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 --2- 172.21.15.129:0/2663483672 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb460072d20 0x7fb46019e8f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.543+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 >> 172.21.15.129:0/2663483672 conn(0x7fb46006ece0 msgr2=0x7fb460115fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.544+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 shutdown_connections 2024-09-05T01:31:29.545 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:29.544+0000 7fb467a04640 1 -- 172.21.15.129:0/2663483672 wait complete. 2024-09-05T01:31:30.066 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:29 smithi129 ceph-mon[30979]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 980 op/s 2024-09-05T01:31:30.066 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:29 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2649248371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:30.120 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T01:31:30.120 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T01:31:30.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 865.0 s, 793.4 tps, lat 123.930 ms stddev 93.790 2024-09-05T01:31:30.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:29 smithi194 ceph-mon[34941]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 980 op/s 2024-09-05T01:31:30.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:29 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2649248371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:29 smithi183 ceph-mon[34589]: pgmap v838: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 980 op/s 2024-09-05T01:31:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:29 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2649248371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:30.610 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.202+0000 7f94b0c86640 1 -- 172.21.15.129:0/3849445115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a40977a0 msgr2=0x7f94a4097c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.202+0000 7f94b0c86640 1 --2- 172.21.15.129:0/3849445115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a40977a0 0x7f94a4097c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9498009900 tx=0x7f949802a410 comp rx=0 tx=0).stop 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 -- 172.21.15.129:0/3849445115 shutdown_connections 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 --2- 172.21.15.129:0/3849445115 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94a4098140 0x7f94a40a4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 --2- 172.21.15.129:0/3849445115 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a40977a0 0x7f94a4097c00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.204 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 --2- 172.21.15.129:0/3849445115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94a409d7a0 0x7f94a409db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 -- 172.21.15.129:0/3849445115 >> 172.21.15.129:0/3849445115 conn(0x7f94a4093440 msgr2=0x7f94a4095860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 -- 172.21.15.129:0/3849445115 shutdown_connections 2024-09-05T01:31:31.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.203+0000 7f94b0c86640 1 -- 172.21.15.129:0/3849445115 wait complete. 2024-09-05T01:31:31.205 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.205+0000 7f94b0c86640 1 Processor -- start 2024-09-05T01:31:31.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.205+0000 7f94b0c86640 1 -- start start 2024-09-05T01:31:31.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94a40977a0 0x7f94a412e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94a4098140 0x7f94a412e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40a7aa0 con 0x7f94a40977a0 2024-09-05T01:31:31.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40a7920 con 0x7f94a409d7a0 2024-09-05T01:31:31.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94b0c86640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40a7c20 con 0x7f94a4098140 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94abfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:55604/0 (socket says 172.21.15.129:55604) 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.206+0000 7f94abfff640 1 -- 172.21.15.129:0/2415609381 learned_addr learned my addr 172.21.15.129:0/2415609381 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 -- 172.21.15.129:0/2415609381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94a4098140 msgr2=0x7f94a412e570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94a4098140 0x7f94a412e570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 -- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94a40977a0 msgr2=0x7f94a412e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94a40977a0 0x7f94a412e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.208 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 -- 172.21.15.129:0/2415609381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a4132fe0 con 0x7f94a409d7a0 2024-09-05T01:31:31.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.207+0000 7f94abfff640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0518b0 tx=0x7f94ac06b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.208+0000 7f94a8ff9640 1 -- 172.21.15.129:0/2415609381 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac074ca0 con 0x7f94a409d7a0 2024-09-05T01:31:31.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.208+0000 7f94a8ff9640 1 -- 172.21.15.129:0/2415609381 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94ac084020 con 0x7f94a409d7a0 2024-09-05T01:31:31.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.208+0000 7f94a8ff9640 1 -- 172.21.15.129:0/2415609381 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac07e6b0 con 0x7f94a409d7a0 2024-09-05T01:31:31.209 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.208+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a41332d0 con 0x7f94a409d7a0 2024-09-05T01:31:31.210 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.208+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a4133730 con 0x7f94a409d7a0 2024-09-05T01:31:31.213 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.212+0000 7f94a8ff9640 1 -- 172.21.15.129:0/2415609381 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f94ac07faa0 con 0x7f94a409d7a0 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.214+0000 7f94a8ff9640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 0x7f949c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.214+0000 7f94a8ff9640 1 -- 172.21.15.129:0/2415609381 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94ac13c550 con 0x7f94a409d7a0 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.214+0000 7f94b42c1740 1 Processor -- start 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.214+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 0x7f949c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.214+0000 7f94b42c1740 1 -- start start 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb6cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.216 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b4cad22a0 con 0x563b4cb802f0 2024-09-05T01:31:31.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b4ca79b10 con 0x563b4cb6c800 2024-09-05T01:31:31.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94b42c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b4cace260 con 0x563b4cb6d120 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 0x7f949c0a5250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005d80 tx=0x7f94a0005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36456/0 (socket says 172.21.15.129:36456) 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94aaffd640 1 -- 172.21.15.129:0/3171697850 learned_addr learned my addr 172.21.15.129:0/3171697850 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb6cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.215+0000 7f94abfff640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 -- 172.21.15.129:0/3171697850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 msgr2=0x563b4cb6cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb6cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 -- 172.21.15.129:0/3171697850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 msgr2=0x563b4cb826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 -- 172.21.15.129:0/3171697850 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4c843b90 con 0x563b4cb6d120 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94abfff640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f94aaffd640 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f949802a980 tx=0x7f94980040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f837902a300bf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f9495ffb640 1 -- 172.21.15.129:0/3171697850 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949803e030 con 0x563b4cb6d120 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.216+0000 7f9495ffb640 1 -- 172.21.15.129:0/3171697850 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9498004260 con 0x563b4cb6d120 2024-09-05T01:31:31.219 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f9495ffb640 1 -- 172.21.15.129:0/3171697850 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498035040 con 0x563b4cb6d120 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f94b42c1740 1 -- 172.21.15.129:0/3171697850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 msgr2=0x563b4cb7fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f949802a980 tx=0x7f94980040f0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f94b42c1740 1 -- 172.21.15.129:0/3171697850 shutdown_connections 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.217+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb7fdb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.218+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3171697850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb6cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.218+0000 7f94b42c1740 1 -- 172.21.15.129:0/3171697850 >> 172.21.15.129:0/3171697850 conn(0x563b4cb68e90 msgr2=0x563b4cb69270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.220 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.218+0000 7f94b42c1740 1 -- 172.21.15.129:0/3171697850 shutdown_connections 2024-09-05T01:31:31.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.218+0000 7f94b42c1740 1 -- 172.21.15.129:0/3171697850 wait complete. 2024-09-05T01:31:31.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.219+0000 7f94b42c1740 1 -- start start 2024-09-05T01:31:31.221 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.219+0000 7f94b42c1740 1 Processor -- start 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6c800 0x563b4cb7d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb802f0 0x563b4cb7dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563b4cb82f10 con 0x563b4cb6d120 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563b4ca5a6e0 con 0x563b4cb802f0 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563b4cb83090 con 0x563b4cb6c800 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94b42c1740 10 client.0 _ll_register_callbacks cb 0x563b4ca7ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48280/0 (socket says 172.21.15.129:48280) 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.220+0000 7f94aaffd640 1 -- 172.21.15.129:0/1830332459 learned_addr learned my addr 172.21.15.129:0/1830332459 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:31.223 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94abfff640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb802f0 0x563b4cb7dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6c800 0x563b4cb7d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 -- 172.21.15.129:0/1830332459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6c800 msgr2=0x563b4cb7d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6c800 0x563b4cb7d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 -- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb802f0 msgr2=0x563b4cb7dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb802f0 0x563b4cb7dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 -- 172.21.15.129:0/1830332459 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4cb8c490 con 0x563b4cb6d120 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f94aaffd640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9498003800 tx=0x7f9498003830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bae06e46e3292 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f948affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.221+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949803e040 con 0x563b4cb6d120 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.222+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9498004260 con 0x563b4cb6d120 2024-09-05T01:31:31.224 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.222+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9498003bd0 con 0x563b4cb6d120 2024-09-05T01:31:31.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.222+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9498035040 con 0x563b4cb6d120 2024-09-05T01:31:31.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 10 client.17049 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:31.225 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563b4cb8ed90 con 0x563b4cb6d120 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f949804c930 con 0x563b4cb6d120 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9498031740 con 0x563b4cb6d120 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 10 client.17049 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 10 client.17049 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 20 client.17049 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cb900e0 0x563b4cb924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 4 client.17049 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.223+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9498004260 con 0x563b4cb900e0 2024-09-05T01:31:31.226 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.225+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cb900e0 0x563b4cb924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.226+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cb900e0 0x563b4cb924d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18162b364e85bb9d server_cookie=770fc0b87a2de134 in_seq=0 out_seq=0 2024-09-05T01:31:31.227 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.226+0000 7f948affd640 10 client.17049 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:31.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.227+0000 7f948affd640 1 -- 172.21.15.129:0/1830332459 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9498004260 con 0x563b4cb900e0 2024-09-05T01:31:31.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.227+0000 7f948affd640 10 client.17049 handle_command_reply: tid=0 2024-09-05T01:31:31.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.227+0000 7f94b42c1740 1 client.17049 shutdown 2024-09-05T01:31:31.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.227+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 msgr2=0x563b4cb7d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.228 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.227+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9498003800 tx=0x7f9498003830 comp rx=0 tx=0).stop 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 shutdown_connections 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cb900e0 0x563b4cb924d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb802f0 0x563b4cb7dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb7d6f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 --2- 172.21.15.129:0/1830332459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6c800 0x563b4cb7d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 >> 172.21.15.129:0/1830332459 conn(0x563b4cb68e90 msgr2=0x563b4cb6ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f948affd640 0 client.17049 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:31.229 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 shutdown_connections 2024-09-05T01:31:31.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 1 -- 172.21.15.129:0/1830332459 wait complete. 2024-09-05T01:31:31.230 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.228+0000 7f94b42c1740 20 client.17049 trim_cache size 0 max 16384 2024-09-05T01:31:31.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.261+0000 7f94b42c1740 1 Processor -- start 2024-09-05T01:31:31.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.261+0000 7f94b42c1740 1 -- start start 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb7da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb95650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4143020 con 0x563b4cb802f0 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94a41408f0 con 0x563b4cb6c800 2024-09-05T01:31:31.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94b42c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94a413e780 con 0x563b4cb6d120 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:36464/0 (socket says 172.21.15.129:36464) 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 -- 172.21.15.129:0/4070580605 learned_addr learned my addr 172.21.15.129:0/4070580605 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb7da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 -- 172.21.15.129:0/4070580605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 msgr2=0x563b4cb7da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb7da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 -- 172.21.15.129:0/4070580605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 msgr2=0x563b4cb95650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb95650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.262+0000 7f94aaffd640 1 -- 172.21.15.129:0/4070580605 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b4cb68cc0 con 0x563b4cb6d120 2024-09-05T01:31:31.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.263+0000 7f94aaffd640 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9498005ab0 tx=0x7f94980040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7076a09db67cc1a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.265+0000 7f94957fa640 1 -- 172.21.15.129:0/4070580605 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9498000e60 con 0x563b4cb6d120 2024-09-05T01:31:31.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.265+0000 7f94957fa640 1 -- 172.21.15.129:0/4070580605 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9498003d50 con 0x563b4cb6d120 2024-09-05T01:31:31.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.265+0000 7f94957fa640 1 -- 172.21.15.129:0/4070580605 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949803e040 con 0x563b4cb6d120 2024-09-05T01:31:31.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.267+0000 7f94b42c1740 1 -- 172.21.15.129:0/4070580605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 msgr2=0x563b4cb95110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.267+0000 7f94b42c1740 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9498005ab0 tx=0x7f94980040f0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.269+0000 7f94b42c1740 1 -- 172.21.15.129:0/4070580605 shutdown_connections 2024-09-05T01:31:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.269+0000 7f94b42c1740 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb802f0 0x563b4cb95650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.269+0000 7f94b42c1740 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb6d120 0x563b4cb95110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.269+0000 7f94b42c1740 1 --2- 172.21.15.129:0/4070580605 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb7da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.269+0000 7f94b42c1740 1 -- 172.21.15.129:0/4070580605 >> 172.21.15.129:0/4070580605 conn(0x563b4cb68e90 msgr2=0x563b4cb7f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.271+0000 7f94b42c1740 1 -- 172.21.15.129:0/4070580605 shutdown_connections 2024-09-05T01:31:31.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.272+0000 7f94b42c1740 1 -- 172.21.15.129:0/4070580605 wait complete. 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.272+0000 7f94b42c1740 1 -- start start 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.272+0000 7f94b42c1740 1 Processor -- start 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb92f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb93450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 0x563b4cb93990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94a413edd0 con 0x563b4cb6d120 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4140220 con 0x563b4cb6c800 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f94a413f480 con 0x563b4cb802f0 2024-09-05T01:31:31.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.273+0000 7f94b42c1740 10 client.0 _ll_register_callbacks cb 0x7f9498004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb93450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb93450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:48306/0 (socket says 172.21.15.129:48306) 2024-09-05T01:31:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94aaffd640 1 -- 172.21.15.129:0/3209660023 learned_addr learned my addr 172.21.15.129:0/3209660023 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94ab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb92f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 0x563b4cb93990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 -- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 msgr2=0x563b4cb92f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.277 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb92f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 -- 172.21.15.129:0/3209660023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 msgr2=0x563b4cb93450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb93450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.274+0000 7f94abfff640 1 -- 172.21.15.129:0/3209660023 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b4cc0aab0 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.275+0000 7f94abfff640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 0x563b4cb93990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94ac081000 tx=0x7f94ac108360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852634f10a24f4c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.275+0000 7f948affd640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.275+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac11ce60 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.275+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9498003d50 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.275+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f94ac111320 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.276+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94ac083040 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.276+0000 7f94b42c1740 10 client.26476 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.276+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563b4cc0bda0 con 0x563b4cb802f0 2024-09-05T01:31:31.278 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.276+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9498003d50 con 0x563b4cb802f0 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f94ac107d90 con 0x563b4cb802f0 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 10 client.26476 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 10 client.26476 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 20 client.26476 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cc0c590 0x563b4cc0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 4 client.26476 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T01:31:31.279 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x563b4cadf9d0 con 0x563b4cc0c590 2024-09-05T01:31:31.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.277+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cc0c590 0x563b4cc0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:31.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.282+0000 7f94ab7fe640 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cc0c590 0x563b4cc0e9b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eaebc9f890929d9 server_cookie=17bac5c36f329c4 in_seq=0 out_seq=0 2024-09-05T01:31:31.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.282+0000 7f948affd640 10 client.26476 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:31.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.578+0000 7f948affd640 1 -- 172.21.15.129:0/3209660023 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563b4cadf9d0 con 0x563b4cc0c590 2024-09-05T01:31:31.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.578+0000 7f948affd640 10 client.26476 handle_command_reply: tid=0 2024-09-05T01:31:31.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.578+0000 7f94b42c1740 1 client.26476 shutdown 2024-09-05T01:31:31.580 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 msgr2=0x563b4cb93990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 0x563b4cb93990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94ac081000 tx=0x7f94ac108360 comp rx=0 tx=0).stop 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 shutdown_connections 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563b4cc0c590 0x563b4cc0e9b0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563b4cb802f0 0x563b4cb93990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563b4cb6d120 0x563b4cb93450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f948affd640 0 client.26476 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 --2- 172.21.15.129:0/3209660023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563b4cb6c800 0x563b4cb92f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 >> 172.21.15.129:0/3209660023 conn(0x563b4cb68e90 msgr2=0x563b4cb8b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 shutdown_connections 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 1 -- 172.21.15.129:0/3209660023 wait complete. 2024-09-05T01:31:31.581 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.579+0000 7f94b42c1740 20 client.26476 trim_cache size 0 max 16384 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 62, 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "efbee404": { 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 62, 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:31:31.582 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:31.583 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 msgr2=0x7f949c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 0x7f949c0a5250 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f94a0005d80 tx=0x7f94a0005c90 comp rx=0 tx=0).stop 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 msgr2=0x7f94a4132900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0518b0 tx=0x7f94ac06b9f0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.584 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 shutdown_connections 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f949c0a2d90 0x7f949c0a5250 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f94a409d7a0 0x7f94a4132900 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f94a4098140 0x7f94a412e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 --2- 172.21.15.129:0/2415609381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94a40977a0 0x7f94a412e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 >> 172.21.15.129:0/2415609381 conn(0x7f94a4093440 msgr2=0x7f94a4093d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 shutdown_connections 2024-09-05T01:31:31.585 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:31.580+0000 7f94b0c86640 1 -- 172.21.15.129:0/2415609381 wait complete. 2024-09-05T01:31:32.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:31 smithi129 ceph-mon[30979]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 872 op/s 2024-09-05T01:31:32.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:31 smithi194 ceph-mon[34941]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 872 op/s 2024-09-05T01:31:32.402 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'efbee404', quiesce took 4.6 seconds. Will release after: 25.4 2024-09-05T01:31:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:31 smithi183 ceph-mon[34589]: pgmap v839: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 13 MiB/s wr, 872 op/s 2024-09-05T01:31:34.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:33 smithi129 ceph-mon[30979]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:31:34.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:33 smithi194 ceph-mon[34941]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:31:34.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:33 smithi183 ceph-mon[34589]: pgmap v840: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 877 op/s 2024-09-05T01:31:35.210 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 870.0 s, 172.9 tps, lat 133.890 ms stddev 99.574 2024-09-05T01:31:36.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:35 smithi129 ceph-mon[30979]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-05T01:31:36.252 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:35 smithi194 ceph-mon[34941]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-05T01:31:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:35 smithi183 ceph-mon[34589]: pgmap v841: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 9.6 MiB/s wr, 775 op/s 2024-09-05T01:31:38.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:37 smithi129 ceph-mon[30979]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.1 MiB/s wr, 683 op/s 2024-09-05T01:31:38.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:37 smithi194 ceph-mon[34941]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.1 MiB/s wr, 683 op/s 2024-09-05T01:31:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:37 smithi183 ceph-mon[34589]: pgmap v842: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 9.1 MiB/s wr, 683 op/s 2024-09-05T01:31:40.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 875.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:31:40.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:39 smithi129 ceph-mon[30979]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 585 op/s 2024-09-05T01:31:40.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:39 smithi194 ceph-mon[34941]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 585 op/s 2024-09-05T01:31:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:39 smithi183 ceph-mon[34589]: pgmap v843: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 8.0 MiB/s wr, 585 op/s 2024-09-05T01:31:42.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:41 smithi129 ceph-mon[30979]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-09-05T01:31:42.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:41 smithi194 ceph-mon[34941]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-09-05T01:31:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:41 smithi183 ceph-mon[34589]: pgmap v844: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 366 op/s 2024-09-05T01:31:42.659 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 -- 172.21.15.129:0/1477023459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33240ffba0 msgr2=0x7f3324100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 --2- 172.21.15.129:0/1477023459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33240ffba0 0x7f3324100020 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f330c009930 tx=0x7f330c02a420 comp rx=0 tx=0).stop 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 -- 172.21.15.129:0/1477023459 shutdown_connections 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 --2- 172.21.15.129:0/1477023459 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 0x7f332410a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 --2- 172.21.15.129:0/1477023459 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33240ffba0 0x7f3324100020 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 --2- 172.21.15.129:0/1477023459 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3324103130 0x7f3324103530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 -- 172.21.15.129:0/1477023459 >> 172.21.15.129:0/1477023459 conn(0x7f33240fb970 msgr2=0x7f33240fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 -- 172.21.15.129:0/1477023459 shutdown_connections 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.658+0000 7f33294e6640 1 -- 172.21.15.129:0/1477023459 wait complete. 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 Processor -- start 2024-09-05T01:31:42.660 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 -- start start 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 0x7f3324197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 0x7f3324198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f332410d6a0 con 0x7f3324103130 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f332410d520 con 0x7f33240ffba0 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33294e6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f332410d820 con 0x7f3324100560 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33237fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46524/0 (socket says 172.21.15.129:46524) 2024-09-05T01:31:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.659+0000 7f33237fe640 1 -- 172.21.15.129:0/247015873 learned_addr learned my addr 172.21.15.129:0/247015873 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f3322ffd640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 0x7f3324197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 -- 172.21.15.129:0/247015873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 msgr2=0x7f3324198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f331bfff640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 0x7f3324198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 0x7f3324198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 -- 172.21.15.129:0/247015873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 msgr2=0x7f3324197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 0x7f3324197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 -- 172.21.15.129:0/247015873 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f332419f960 con 0x7f3324103130 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f3322ffd640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 0x7f3324197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33237fe640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f331c00cad0 tx=0x7f331c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f3320ff9640 1 -- 172.21.15.129:0/247015873 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c00ae20 con 0x7f3324103130 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f332419fc50 con 0x7f3324103130 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f3320ff9640 1 -- 172.21.15.129:0/247015873 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f331c005020 con 0x7f3324103130 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f3320ff9640 1 -- 172.21.15.129:0/247015873 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c00f6b0 con 0x7f3324103130 2024-09-05T01:31:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.660+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33241a01a0 con 0x7f3324103130 2024-09-05T01:31:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f3320ff9640 1 -- 172.21.15.129:0/247015873 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f331c0051d0 con 0x7f3324103130 2024-09-05T01:31:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 Processor -- start 2024-09-05T01:31:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 -- start start 2024-09-05T01:31:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b311da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b314830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d6b2702f0 con 0x563d6b30c9a0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d6b218270 con 0x563d6b312370 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.663+0000 7f332c277740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d6b26c2b0 con 0x563d6b30d2e0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f3322ffd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b311da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46534/0 (socket says 172.21.15.129:46534) 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 -- 172.21.15.129:0/4273250818 learned_addr learned my addr 172.21.15.129:0/4273250818 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f3320ff9640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 0x7f33100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b314830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f33237fe640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 0x7f33100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f3320ff9640 1 -- 172.21.15.129:0/247015873 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f331c0cf5d0 con 0x7f3324103130 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 -- 172.21.15.129:0/4273250818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 msgr2=0x563d6b311da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b311da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 -- 172.21.15.129:0/4273250818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 msgr2=0x563d6b314830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b314830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 -- 172.21.15.129:0/4273250818 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6b217340 con 0x563d6b30c9a0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f3322ffd640 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b311da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f331bfff640 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f330c005940 tx=0x7f330c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59685d4ebfebc825 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f33237fe640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 0x7f33100a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f331c09b9b0 tx=0x7f331c090e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f33197fa640 1 -- 172.21.15.129:0/4273250818 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c035050 con 0x563d6b30c9a0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f33197fa640 1 -- 172.21.15.129:0/4273250818 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f330c0044d0 con 0x563d6b30c9a0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.664+0000 7f33197fa640 1 -- 172.21.15.129:0/4273250818 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c004920 con 0x563d6b30c9a0 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 -- 172.21.15.129:0/4273250818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 msgr2=0x563d6b30cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f330c005940 tx=0x7f330c005a20 comp rx=0 tx=0).stop 2024-09-05T01:31:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 -- 172.21.15.129:0/4273250818 shutdown_connections 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b314830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b311da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 --2- 172.21.15.129:0/4273250818 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b30cda0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 -- 172.21.15.129:0/4273250818 >> 172.21.15.129:0/4273250818 conn(0x563d6b308010 msgr2=0x563d6b308410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 -- 172.21.15.129:0/4273250818 shutdown_connections 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.665+0000 7f332c277740 1 -- 172.21.15.129:0/4273250818 wait complete. 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 -- start start 2024-09-05T01:31:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 Processor -- start 2024-09-05T01:31:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 0x563d6b3170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b312370 0x563d6b31e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563d6b3152e0 con 0x563d6b30c9a0 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563d6b1f9290 con 0x563d6b30d2e0 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563d6b315460 con 0x563d6b312370 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f332c277740 10 client.0 _ll_register_callbacks cb 0x563d6b22a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46546/0 (socket says 172.21.15.129:46546) 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 -- 172.21.15.129:0/284880898 learned_addr learned my addr 172.21.15.129:0/284880898 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f3322ffd640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 0x563d6b3170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 -- 172.21.15.129:0/284880898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b312370 msgr2=0x563d6b31e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b312370 0x563d6b31e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 -- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 msgr2=0x563d6b3170d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 0x563d6b3170d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f331bfff640 1 -- 172.21.15.129:0/284880898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6af83210 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.666+0000 7f3322ffd640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 0x563d6b3170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f331bfff640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f330c009a20 tx=0x7f330c02aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58797f64bf50e38c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c04e060 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f330c003a00 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330c0044d0 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f330c045040 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 10 client.17064 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d6b04f620 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f330c0044d0 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f330c049c00 con 0x563d6b30c9a0 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 10 client.17064 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 20 client.17064 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b322410 0x563d6b324800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 4 client.17064 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.667+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6b27da20 con 0x563d6b322410 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.668+0000 7f3322ffd640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b322410 0x563d6b324800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.668+0000 7f3322ffd640 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b322410 0x563d6b324800 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef273b6134bd947c server_cookie=e1a27eb75ff69f in_seq=0 out_seq=0 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.668+0000 7f330a7fc640 10 client.17064 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f330a7fc640 1 -- 172.21.15.129:0/284880898 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d6b27da20 con 0x563d6b322410 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f330a7fc640 10 client.17064 handle_command_reply: tid=0 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 client.17064 shutdown 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 msgr2=0x563d6b316b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f330c009a20 tx=0x7f330c02aad0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 shutdown_connections 2024-09-05T01:31:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b322410 0x563d6b324800 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b312370 0x563d6b31e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f330a7fc640 0 client.17064 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b30d2e0 0x563d6b3170d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 --2- 172.21.15.129:0/284880898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b316b90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 >> 172.21.15.129:0/284880898 conn(0x563d6b308010 msgr2=0x563d6b309820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 shutdown_connections 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 1 -- 172.21.15.129:0/284880898 wait complete. 2024-09-05T01:31:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.669+0000 7f332c277740 20 client.17064 trim_cache size 0 max 16384 2024-09-05T01:31:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.683+0000 7f332c277740 1 Processor -- start 2024-09-05T01:31:42.684 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 -- start start 2024-09-05T01:31:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.688 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b308ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b309400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33241ad050 con 0x563d6b30c9a0 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33241aa920 con 0x563d6b312370 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f332c277740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33241a8770 con 0x563d6b30d2e0 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46552/0 (socket says 172.21.15.129:46552) 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 -- 172.21.15.129:0/2596929975 learned_addr learned my addr 172.21.15.129:0/2596929975 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f3322ffd640 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b308ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 -- 172.21.15.129:0/2596929975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 msgr2=0x563d6b308ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b308ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 -- 172.21.15.129:0/2596929975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 msgr2=0x563d6b309400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.689 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b309400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 -- 172.21.15.129:0/2596929975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6b31f5b0 con 0x563d6b30c9a0 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.684+0000 7f331bfff640 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c2a063aa580295 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f3318ff9640 1 -- 172.21.15.129:0/2596929975 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c05e070 con 0x563d6b30c9a0 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f3318ff9640 1 -- 172.21.15.129:0/2596929975 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f330c041070 con 0x563d6b30c9a0 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f3318ff9640 1 -- 172.21.15.129:0/2596929975 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c04e070 con 0x563d6b30c9a0 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 -- 172.21.15.129:0/2596929975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 msgr2=0x563d6b317050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f330c000c00 tx=0x7f330c009800 comp rx=0 tx=0).stop 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 -- 172.21.15.129:0/2596929975 shutdown_connections 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b309400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b308ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 --2- 172.21.15.129:0/2596929975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b317050 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 -- 172.21.15.129:0/2596929975 >> 172.21.15.129:0/2596929975 conn(0x563d6b308010 msgr2=0x563d6b316940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 -- 172.21.15.129:0/2596929975 shutdown_connections 2024-09-05T01:31:42.690 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.685+0000 7f332c277740 1 -- 172.21.15.129:0/2596929975 wait complete. 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 -- start start 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 Processor -- start 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b388730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b38f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33241a8de0 con 0x563d6b30c9a0 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f33241aa250 con 0x563d6b312370 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f33241a94b0 con 0x563d6b30d2e0 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f332c277740 10 client.0 _ll_register_callbacks cb 0x7f3314009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46558/0 (socket says 172.21.15.129:46558) 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 -- 172.21.15.129:0/3874560910 learned_addr learned my addr 172.21.15.129:0/3874560910 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:42.691 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f3322ffd640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b388730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 -- 172.21.15.129:0/3874560910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 msgr2=0x563d6b388730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b388730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 -- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 msgr2=0x563d6b38f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b38f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.686+0000 7f331bfff640 1 -- 172.21.15.129:0/3874560910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6b38fd80 con 0x563d6b30c9a0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f331bfff640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f330c009a20 tx=0x7f330c039f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e1e36376d04e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c04e030 con 0x563d6b30c9a0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f330c031900 con 0x563d6b30c9a0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f330c041070 con 0x563d6b30c9a0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f330c05e070 con 0x563d6b30c9a0 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 10 client.17070 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:42.692 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d6b391090 con 0x563d6b30c9a0 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f330c041070 con 0x563d6b30c9a0 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f330c049420 con 0x563d6b30c9a0 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 20 client.17070 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b391830 0x563d6b393c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 4 client.17070 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.687+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d6b277d70 con 0x563d6b391830 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.688+0000 7f3322ffd640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b391830 0x563d6b393c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.688+0000 7f330a7fc640 10 client.17070 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:42.693 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.688+0000 7f3322ffd640 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b391830 0x563d6b393c50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d118608becc547 server_cookie=948483703d92f3e8 in_seq=0 out_seq=0 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.701+0000 7f330a7fc640 1 -- 172.21.15.129:0/3874560910 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d6b277d70 con 0x563d6b391830 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.701+0000 7f330a7fc640 10 client.17070 handle_command_reply: tid=0 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.701+0000 7f332c277740 1 client.17070 shutdown 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 msgr2=0x563d6b3881f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f330c009a20 tx=0x7f330c039f50 comp rx=0 tx=0).stop 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 shutdown_connections 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563d6b391830 0x563d6b393c50 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563d6b312370 0x563d6b38f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.703 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563d6b30d2e0 0x563d6b388730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f330a7fc640 0 client.17070 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 --2- 172.21.15.129:0/3874560910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563d6b30c9a0 0x563d6b3881f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 >> 172.21.15.129:0/3874560910 conn(0x563d6b308010 msgr2=0x563d6b3069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 shutdown_connections 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 1 -- 172.21.15.129:0/3874560910 wait complete. 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f332c277740 20 client.17070 trim_cache size 0 max 16384 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:31:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:31:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:31:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 msgr2=0x7f33100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 0x7f33100a50f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f331c09b9b0 tx=0x7f331c090e70 comp rx=0 tx=0).stop 2024-09-05T01:31:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 msgr2=0x7f332419f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f331c00cad0 tx=0x7f331c00cf90 comp rx=0 tx=0).stop 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 shutdown_connections 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f33100a2c30 0x7f33100a50f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3324103130 0x7f332419f2c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3324100560 0x7f3324198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.702+0000 7f33294e6640 1 --2- 172.21.15.129:0/247015873 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f33240ffba0 0x7f3324197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.703+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 >> 172.21.15.129:0/247015873 conn(0x7f33240fb970 msgr2=0x7f3324106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.703+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 shutdown_connections 2024-09-05T01:31:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:42.703+0000 7f33294e6640 1 -- 172.21.15.129:0/247015873 wait complete. 2024-09-05T01:31:44.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:43 smithi129 ceph-mon[30979]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-09-05T01:31:44.251 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:43 smithi194 ceph-mon[34941]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-09-05T01:31:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:43 smithi183 ceph-mon[34589]: pgmap v845: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.1 MiB/s wr, 264 op/s 2024-09-05T01:31:45.212 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 880.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:31:46.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:45 smithi129 ceph-mon[30979]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-05T01:31:46.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:45 smithi194 ceph-mon[34941]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-05T01:31:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:45 smithi183 ceph-mon[34589]: pgmap v846: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 151 op/s 2024-09-05T01:31:48.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:47 smithi129 ceph-mon[30979]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 860 KiB/s wr, 71 op/s 2024-09-05T01:31:48.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:47 smithi194 ceph-mon[34941]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 860 KiB/s wr, 71 op/s 2024-09-05T01:31:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:47 smithi183 ceph-mon[34589]: pgmap v847: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 860 KiB/s wr, 71 op/s 2024-09-05T01:31:50.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 885.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:31:50.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:49 smithi194 ceph-mon[34941]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 209 KiB/s wr, 36 op/s 2024-09-05T01:31:50.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:49 smithi183 ceph-mon[34589]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 209 KiB/s wr, 36 op/s 2024-09-05T01:31:50.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:49 smithi129 ceph-mon[30979]: pgmap v848: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 209 KiB/s wr, 36 op/s 2024-09-05T01:31:52.250 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:51 smithi194 ceph-mon[34941]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 183 KiB/s wr, 31 op/s 2024-09-05T01:31:52.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:51 smithi183 ceph-mon[34589]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 183 KiB/s wr, 31 op/s 2024-09-05T01:31:52.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:51 smithi129 ceph-mon[30979]: pgmap v849: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 183 KiB/s wr, 31 op/s 2024-09-05T01:31:54.294 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:54 smithi129 ceph-mon[30979]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 205 KiB/s wr, 34 op/s 2024-09-05T01:31:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:54 smithi183 ceph-mon[34589]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 205 KiB/s wr, 34 op/s 2024-09-05T01:31:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:54 smithi194 ceph-mon[34941]: pgmap v850: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 205 KiB/s wr, 34 op/s 2024-09-05T01:31:55.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:31:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- 172.21.15.129:0/1923834745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf681025c0 msgr2=0x7fcf681049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/1923834745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf681025c0 0x7fcf681049b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009900 tx=0x7fcf5802a410 comp rx=0 tx=0).stop 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- 172.21.15.129:0/1923834745 shutdown_connections 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/1923834745 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 0x7fcf68109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/1923834745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf68104ef0 0x7fcf681072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/1923834745 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf681025c0 0x7fcf681049b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- 172.21.15.129:0/1923834745 >> 172.21.15.129:0/1923834745 conn(0x7fcf680fc1e0 msgr2=0x7fcf680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- 172.21.15.129:0/1923834745 shutdown_connections 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- 172.21.15.129:0/1923834745 wait complete. 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 Processor -- start 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.803+0000 7fcf6e59a640 1 -- start start 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 0x7fcf68197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 0x7fcf6819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810d6e0 con 0x7fcf68104ef0 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810d560 con 0x7fcf681025c0 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6e59a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf6810d860 con 0x7fcf68107820 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45734/0 (socket says 172.21.15.129:45734) 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 -- 172.21.15.129:0/169454978 learned_addr learned my addr 172.21.15.129:0/169454978 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf67fff640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 0x7fcf68197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 0x7fcf6819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 -- 172.21.15.129:0/169454978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 msgr2=0x7fcf6819f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 0x7fcf6819f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 -- 172.21.15.129:0/169454978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 msgr2=0x7fcf68197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 0x7fcf68197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf677fe640 1 -- 172.21.15.129:0/169454978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6819f980 con 0x7fcf68104ef0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.804+0000 7fcf67fff640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 0x7fcf68197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf677fe640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00d770 tx=0x7fcf5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf657fa640 1 -- 172.21.15.129:0/169454978 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c017070 con 0x7fcf68104ef0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf657fa640 1 -- 172.21.15.129:0/169454978 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf5c009c30 con 0x7fcf68104ef0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6819fc70 con 0x7fcf68104ef0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf657fa640 1 -- 172.21.15.129:0/169454978 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00f620 con 0x7fcf68104ef0 2024-09-05T01:31:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.805+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf681a01c0 con 0x7fcf68104ef0 2024-09-05T01:31:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.807+0000 7fcf657fa640 1 -- 172.21.15.129:0/169454978 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fcf5c01a070 con 0x7fcf68104ef0 2024-09-05T01:31:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf657fa640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 0x7fcf500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf67fff640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 0x7fcf500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf657fa640 1 -- 172.21.15.129:0/169454978 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcf5c0cf030 con 0x7fcf68104ef0 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf67fff640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 0x7fcf500a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcf580314e0 tx=0x7fcf58032c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf7140e740 1 Processor -- start 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf7140e740 1 -- start start 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e29a0 0x563e896e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 0x563e896e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.808+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf7140e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563e896462f0 con 0x563e896e8370 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf7140e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e895ee270 con 0x563e896e29a0 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf7140e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e896422b0 con 0x563e896e32e0 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45746/0 (socket says 172.21.15.129:45746) 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 -- 172.21.15.129:0/3585584373 learned_addr learned my addr 172.21.15.129:0/3585584373 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf67fff640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e29a0 0x563e896e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf677fe640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 0x563e896e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 -- 172.21.15.129:0/3585584373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 msgr2=0x563e896e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 0x563e896e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 -- 172.21.15.129:0/3585584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e29a0 msgr2=0x563e896e2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e29a0 0x563e896e2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 -- 172.21.15.129:0/3585584373 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e895ed340 con 0x563e896e8370 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf677fe640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 0x563e896e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009f80 tx=0x7fcf60010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b894258d2a168af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf4e7fc640 1 -- 172.21.15.129:0/3585584373 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60003b70 con 0x563e896e8370 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf4e7fc640 1 -- 172.21.15.129:0/3585584373 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf60003d10 con 0x563e896e8370 2024-09-05T01:31:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.809+0000 7fcf4e7fc640 1 -- 172.21.15.129:0/3585584373 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60011ad0 con 0x563e896e8370 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- 172.21.15.129:0/3585584373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 msgr2=0x563e896ea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcf60009f80 tx=0x7fcf60010860 comp rx=0 tx=0).stop 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- 172.21.15.129:0/3585584373 shutdown_connections 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e8370 0x563e896ea830 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e32e0 0x563e896e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 --2- 172.21.15.129:0/3585584373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e29a0 0x563e896e2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.823 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- 172.21.15.129:0/3585584373 >> 172.21.15.129:0/3585584373 conn(0x563e896de010 msgr2=0x563e896de410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- 172.21.15.129:0/3585584373 shutdown_connections 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- 172.21.15.129:0/3585584373 wait complete. 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.810+0000 7fcf7140e740 1 -- start start 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 Processor -- start 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896ecc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 0x563e896f4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.824 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563e896eb340 con 0x563e896e32e0 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563e895cf290 con 0x563e896e8370 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x563e896eb4c0 con 0x563e896e29a0 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf7140e740 10 client.0 _ll_register_callbacks cb 0x563e89600140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45748/0 (socket says 172.21.15.129:45748) 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 -- 172.21.15.129:0/972195053 learned_addr learned my addr 172.21.15.129:0/972195053 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf67fff640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896ecc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.825 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 0x563e896f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 -- 172.21.15.129:0/972195053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 msgr2=0x563e896ecc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896ecc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 -- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 msgr2=0x563e896f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 0x563e896f4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 -- 172.21.15.129:0/972195053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e89359210 con 0x563e896e32e0 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 0x563e896f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf677fe640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00d740 tx=0x7fcf5c09aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c9aa183c58d162 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf477fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c021070 con 0x563e896e32e0 2024-09-05T01:31:55.826 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.811+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcf5c0ccbe0 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf60003d10 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf5c09b480 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 10 client.17082 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563e89425620 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf60003d10 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcf5c0971d0 con 0x563e896e32e0 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 10 client.17082 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 20 client.17082 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896f8790 0x563e896fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 4 client.17082 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.812+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e89653a20 con 0x563e896f8790 2024-09-05T01:31:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.813+0000 7fcf67fff640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896f8790 0x563e896fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.813+0000 7fcf477fe640 10 client.17082 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.813+0000 7fcf67fff640 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896f8790 0x563e896fab80 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf5df79e66e6599 server_cookie=54c9ddeaceeac480 in_seq=0 out_seq=0 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf477fe640 1 -- 172.21.15.129:0/972195053 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e89653a20 con 0x563e896f8790 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf477fe640 10 client.17082 handle_command_reply: tid=0 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 client.17082 shutdown 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 msgr2=0x563e896ed160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00d740 tx=0x7fcf5c09aea0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 shutdown_connections 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896f8790 0x563e896fab80 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e8370 0x563e896f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf477fe640 0 client.17082 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896ed160 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 --2- 172.21.15.129:0/972195053 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896ecc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 >> 172.21.15.129:0/972195053 conn(0x563e896de010 msgr2=0x563e896df820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 shutdown_connections 2024-09-05T01:31:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 1 -- 172.21.15.129:0/972195053 wait complete. 2024-09-05T01:31:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.814+0000 7fcf7140e740 20 client.17082 trim_cache size 0 max 16384 2024-09-05T01:31:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.843+0000 7fcf7140e740 1 Processor -- start 2024-09-05T01:31:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.843+0000 7fcf7140e740 1 -- start start 2024-09-05T01:31:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.847 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 0x563e896ff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681ad140 con 0x563e896de010 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681aaa10 con 0x563e896e32e0 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf7140e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681a8860 con 0x563e896e29a0 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45752/0 (socket says 172.21.15.129:45752) 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 -- 172.21.15.129:0/1460305841 learned_addr learned my addr 172.21.15.129:0/1460305841 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 0x563e896ff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf677fe640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 -- 172.21.15.129:0/1460305841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 msgr2=0x563e896f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 -- 172.21.15.129:0/1460305841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 msgr2=0x563e896ff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.848 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 0x563e896ff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 -- 172.21.15.129:0/1460305841 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e896faef0 con 0x563e896de010 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 0x563e896ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009900 tx=0x7fcf5803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616330e4e8b2bbb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf4dffb640 1 -- 172.21.15.129:0/1460305841 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58035050 con 0x563e896de010 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf4dffb640 1 -- 172.21.15.129:0/1460305841 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf58005040 con 0x563e896de010 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.844+0000 7fcf4dffb640 1 -- 172.21.15.129:0/1460305841 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58004950 con 0x563e896de010 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- 172.21.15.129:0/1460305841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 msgr2=0x563e896ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009900 tx=0x7fcf5803f000 comp rx=0 tx=0).stop 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- 172.21.15.129:0/1460305841 shutdown_connections 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896e32e0 0x563e896ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1460305841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896de010 0x563e896ff460 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.849 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- 172.21.15.129:0/1460305841 >> 172.21.15.129:0/1460305841 conn(0x563e896e3da0 msgr2=0x563e896dd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- 172.21.15.129:0/1460305841 shutdown_connections 2024-09-05T01:31:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- 172.21.15.129:0/1460305841 wait complete. 2024-09-05T01:31:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.845+0000 7fcf7140e740 1 -- start start 2024-09-05T01:31:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 Processor -- start 2024-09-05T01:31:55.852 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896de010 0x563e896f1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681a8ed0 con 0x563e896e32e0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681aa340 con 0x563e896de010 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fcf681a95a0 con 0x563e896e29a0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf7140e740 10 client.0 _ll_register_callbacks cb 0x7fcf5c010ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45762/0 (socket says 172.21.15.129:45762) 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 -- 172.21.15.129:0/1485298617 learned_addr learned my addr 172.21.15.129:0/1485298617 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf677fe640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896de010 0x563e896f1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 -- 172.21.15.129:0/1485298617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 msgr2=0x563e896f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.853 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 -- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896de010 msgr2=0x563e896f1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896de010 0x563e896f1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.846+0000 7fcf6cb37640 1 -- 172.21.15.129:0/1485298617 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e896f3600 con 0x563e896e32e0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf6cb37640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x563e896dee90 tx=0x7fcf60011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d560d5a5bb551f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60004be0 con 0x563e896e32e0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fcf60004d80 con 0x563e896e32e0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf58005040 con 0x563e896e32e0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcf60005050 con 0x563e896e32e0 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 10 client.17088 fetch_fsmap learned FSMap version 34 2024-09-05T01:31:55.854 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563e89763740 con 0x563e896e32e0 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf58005040 con 0x563e896e32e0 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fcf60029680 con 0x563e896e32e0 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 10 client.17088 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 20 client.17088 populate_metadata read hostname 'smithi129' 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896e8370 0x563e89765fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 4 client.17088 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.847+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e8964dd70 con 0x563e896e8370 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.848+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896e8370 0x563e89765fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.848+0000 7fcf477fe640 10 client.17088 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:55.855 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.848+0000 7fcf67fff640 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896e8370 0x563e89765fe0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfd44578533a6c server_cookie=c4b46b6b2f222830 in_seq=0 out_seq=0 2024-09-05T01:31:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf477fe640 1 -- 172.21.15.129:0/1485298617 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e8964dd70 con 0x563e896e8370 2024-09-05T01:31:55.861 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf477fe640 10 client.17088 handle_command_reply: tid=0 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 client.17088 shutdown 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 msgr2=0x563e896f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x563e896dee90 tx=0x7fcf60011cd0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 shutdown_connections 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x563e896e8370 0x563e89765fe0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e896e32e0 0x563e896f2ff0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf477fe640 0 client.17088 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x563e896e29a0 0x563e896f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 --2- 172.21.15.129:0/1485298617 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563e896de010 0x563e896f1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 >> 172.21.15.129:0/1485298617 conn(0x563e896e3da0 msgr2=0x563e896dc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.862 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 shutdown_connections 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.859+0000 7fcf7140e740 1 -- 172.21.15.129:0/1485298617 wait complete. 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf7140e740 20 client.17088 trim_cache size 0 max 16384 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:31:55.863 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 msgr2=0x7fcf500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 0x7fcf500a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcf580314e0 tx=0x7fcf58032c60 comp rx=0 tx=0).stop 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 msgr2=0x7fcf681982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c00d770 tx=0x7fcf5c00dc30 comp rx=0 tx=0).stop 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 shutdown_connections 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fcf500a2c30 0x7fcf500a50f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.864 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fcf68107820 0x7fcf6819f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf68104ef0 0x7fcf681982a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 --2- 172.21.15.129:0/169454978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcf681025c0 0x7fcf68197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 >> 172.21.15.129:0/169454978 conn(0x7fcf680fc1e0 msgr2=0x7fcf68102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 shutdown_connections 2024-09-05T01:31:55.865 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:31:55.860+0000 7fcf6e59a640 1 -- 172.21.15.129:0/169454978 wait complete. 2024-09-05T01:31:56.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:56 smithi183 ceph-mon[34589]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-09-05T01:31:56.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:56 smithi129 ceph-mon[30979]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-09-05T01:31:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:56 smithi194 ceph-mon[34941]: pgmap v851: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 214 KiB/s wr, 36 op/s 2024-09-05T01:31:57.804 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:31:58.061 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:31:58.182 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:58 smithi129 ceph-mon[30979]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-09-05T01:31:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:58 smithi183 ceph-mon[34589]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-09-05T01:31:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:58 smithi194 ceph-mon[34941]: pgmap v852: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 198 KiB/s wr, 31 op/s 2024-09-05T01:31:58.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- 172.21.15.129:0/1452550398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c102790 msgr2=0x7f1f7c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:58.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 --2- 172.21.15.129:0/1452550398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c102790 0x7f1f7c1067e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009900 tx=0x7f1f6c02a410 comp rx=0 tx=0).stop 2024-09-05T01:31:58.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- 172.21.15.129:0/1452550398 shutdown_connections 2024-09-05T01:31:58.612 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 --2- 172.21.15.129:0/1452550398 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c106f10 0x7f1f7c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 --2- 172.21.15.129:0/1452550398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c102790 0x7f1f7c1067e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 --2- 172.21.15.129:0/1452550398 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 0x7f1f7c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- 172.21.15.129:0/1452550398 >> 172.21.15.129:0/1452550398 conn(0x7f1f7c0fd8c0 msgr2=0x7f1f7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- 172.21.15.129:0/1452550398 shutdown_connections 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- 172.21.15.129:0/1452550398 wait complete. 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 Processor -- start 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.576+0000 7f1f8408f640 1 -- start start 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 0x7f1f7c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c102790 0x7f1f7c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:58.613 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c111e40 con 0x7f1f7c106f10 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c111cc0 con 0x7f1f7c101de0 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c111fc0 con 0x7f1f7c102790 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45778/0 (socket says 172.21.15.129:45778) 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 -- 172.21.15.129:0/2091870487 learned_addr learned my addr 172.21.15.129:0/2091870487 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8162a640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c102790 0x7f1f7c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f81e2b640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 0x7f1f7c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 -- 172.21.15.129:0/2091870487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c102790 msgr2=0x7f1f7c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c102790 0x7f1f7c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 -- 172.21.15.129:0/2091870487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 msgr2=0x7f1f7c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 0x7f1f7c198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 -- 172.21.15.129:0/2091870487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c19d2d0 con 0x7f1f7c106f10 2024-09-05T01:31:58.614 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8262c640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b970 tx=0x7f1f7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74019070 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f74014440 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c19d5c0 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f740075d0 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.577+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c1a4ea0 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.579+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c1021c0 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.579+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f1f7401c070 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.580+0000 7f1f72ffd640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 0x7f1f680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.581+0000 7f1f81e2b640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 0x7f1f680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.581+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f1f740ce9a0 con 0x7f1f7c106f10 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.581+0000 7f1f81e2b640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 0x7f1f680a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005e10 tx=0x7f1f78005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:31:58.615 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.585+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f7409aea0 con 0x7f1f7c106f10 2024-09-05T01:31:58.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.797+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f7c02d7c0 con 0x7f1f7c106f10 2024-09-05T01:31:58.799 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.798+0000 7f1f72ffd640 1 -- 172.21.15.129:0/2091870487 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f1f7409f1a0 con 0x7f1f7c106f10 2024-09-05T01:31:58.800 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:31:58.800 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:31:58.800 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 msgr2=0x7f1f680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 0x7f1f680a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1f78005e10 tx=0x7f1f78005da0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 msgr2=0x7f1f7c19cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400b970 tx=0x7f1f7400be30 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 shutdown_connections 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f1f680a2b80 0x7f1f680a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c106f10 0x7f1f7c19cbf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f1f7c102790 0x7f1f7c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 --2- 172.21.15.129:0/2091870487 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1f7c101de0 0x7f1f7c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:31:58.803 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 >> 172.21.15.129:0/2091870487 conn(0x7f1f7c0fd8c0 msgr2=0x7f1f7c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:31:58.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 shutdown_connections 2024-09-05T01:31:58.804 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:31:58.800+0000 7f1f8408f640 1 -- 172.21.15.129:0/2091870487 wait complete. 2024-09-05T01:31:59.224 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:31:59 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2091870487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:59.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:31:59 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2091870487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:59.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:31:59 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2091870487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:31:59.689 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id efbee404 --release --await' 2024-09-05T01:31:59.689 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id efbee404 --release --await 2024-09-05T01:31:59.941 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:00.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 895.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:00.348 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:00 smithi129 ceph-mon[30979]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 231 KiB/s wr, 38 op/s 2024-09-05T01:32:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:00 smithi183 ceph-mon[34589]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 231 KiB/s wr, 38 op/s 2024-09-05T01:32:00.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 -- 172.21.15.129:0/783256782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 msgr2=0x7f74181067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 --2- 172.21.15.129:0/783256782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f74181067e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7408009900 tx=0x7f740802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:00.466 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 -- 172.21.15.129:0/783256782 shutdown_connections 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 --2- 172.21.15.129:0/783256782 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 0x7f741810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 --2- 172.21.15.129:0/783256782 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f74181067e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 --2- 172.21.15.129:0/783256782 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 0x7f74181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 -- 172.21.15.129:0/783256782 >> 172.21.15.129:0/783256782 conn(0x7f74180fd8c0 msgr2=0x7f74180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 -- 172.21.15.129:0/783256782 shutdown_connections 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.442+0000 7f741fe0a640 1 -- 172.21.15.129:0/783256782 wait complete. 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 Processor -- start 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 -- start start 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 0x7f7418198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 0x7f741819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7418111e40 con 0x7f7418102790 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7418111cc0 con 0x7f7418106f10 2024-09-05T01:32:00.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741fe0a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7418111fc0 con 0x7f7418101de0 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45794/0 (socket says 172.21.15.129:45794) 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 -- 172.21.15.129:0/1172698721 learned_addr learned my addr 172.21.15.129:0/1172698721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741dba6640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 0x7f7418198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 -- 172.21.15.129:0/1172698721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 msgr2=0x7f7418198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 0x7f7418198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741e3a7640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 0x7f741819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 -- 172.21.15.129:0/1172698721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 msgr2=0x7f741819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 0x7f741819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.443+0000 7f741d3a5640 1 -- 172.21.15.129:0/1172698721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741819d3d0 con 0x7f7418102790 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f7408004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f740effd640 1 -- 172.21.15.129:0/1172698721 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740803f030 con 0x7f7418102790 2024-09-05T01:32:00.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f740effd640 1 -- 172.21.15.129:0/1172698721 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f74080035f0 con 0x7f7418102790 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f740effd640 1 -- 172.21.15.129:0/1172698721 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408035070 con 0x7f7418102790 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741819d660 con 0x7f7418102790 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.444+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181a4fa0 con 0x7f7418102790 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.446+0000 7f740effd640 1 -- 172.21.15.129:0/1172698721 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7408003b10 con 0x7f7418102790 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.446+0000 7f74221e3740 1 Processor -- start 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 -- start start 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f740effd640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 0x7f74040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56381780b620 con 0x5638178a6980 2024-09-05T01:32:00.469 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5638177b2e50 con 0x5638178b1240 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f74221e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5638178075e0 con 0x5638178a6060 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741e3a7640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 0x7f74040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45810/0 (socket says 172.21.15.129:45810) 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741d3a5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 -- 172.21.15.129:0/1989704135 learned_addr learned my addr 172.21.15.129:0/1989704135 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741e3a7640 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f740effd640 1 -- 172.21.15.129:0/1172698721 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f74080effd0 con 0x7f7418102790 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 -- 172.21.15.129:0/1989704135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 msgr2=0x5638178a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.470 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 -- 172.21.15.129:0/1989704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 msgr2=0x5638178b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 -- 172.21.15.129:0/1989704135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638177a3d40 con 0x5638178a6980 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741e3a7640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 0x7f74040a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74100148c0 tx=0x7f7410001df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.447+0000 7f741dba6640 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f741400ac20 tx=0x7f741400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac484d56cca6292 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f7403fff640 1 -- 172.21.15.129:0/1989704135 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414021070 con 0x5638178a6980 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f7403fff640 1 -- 172.21.15.129:0/1989704135 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741401c440 con 0x5638178a6980 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f7403fff640 1 -- 172.21.15.129:0/1989704135 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741401a580 con 0x5638178a6980 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 -- 172.21.15.129:0/1989704135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 msgr2=0x5638178b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f741400ac20 tx=0x7f741400f790 comp rx=0 tx=0).stop 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 -- 172.21.15.129:0/1989704135 shutdown_connections 2024-09-05T01:32:00.471 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6980 0x5638178b0d00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 --2- 172.21.15.129:0/1989704135 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.448+0000 7f74221e3740 1 -- 172.21.15.129:0/1989704135 >> 172.21.15.129:0/1989704135 conn(0x5638178a25f0 msgr2=0x5638178a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.449+0000 7f74221e3740 1 -- 172.21.15.129:0/1989704135 shutdown_connections 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.449+0000 7f74221e3740 1 -- 172.21.15.129:0/1989704135 wait complete. 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.449+0000 7f74221e3740 1 -- start start 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.449+0000 7f74221e3740 1 Processor -- start 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.449+0000 7f74221e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6980 0x5638178ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.472 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5638178b3e10 con 0x5638178a6060 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563817793f10 con 0x5638178b1240 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5638178b3f90 con 0x5638178a6980 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 10 client.0 _ll_register_callbacks cb 0x5638177b7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45812/0 (socket says 172.21.15.129:45812) 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 -- 172.21.15.129:0/1477052887 learned_addr learned my addr 172.21.15.129:0/1477052887 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6980 0x5638178ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 -- 172.21.15.129:0/1477052887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6980 msgr2=0x5638178ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6980 0x5638178ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 -- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 msgr2=0x5638178aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.473 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 -- 172.21.15.129:0/1477052887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381751a580 con 0x5638178a6060 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f741dba6640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f741401caa0 tx=0x7f741401cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f262312e4f92daf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74017fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741402b070 con 0x5638178a6060 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7414014a40 con 0x5638178a6060 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741401c440 con 0x5638178a6060 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7414037050 con 0x5638178a6060 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 10 client.17106 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:00.474 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.450+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5638175e4960 con 0x5638178a6060 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741401c440 con 0x5638178a6060 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7414026ad0 con 0x5638178a6060 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 10 client.17106 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 10 client.17106 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 20 client.17106 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5638178c0cb0 0x5638178c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 4 client.17106 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563817818b90 con 0x5638178c0cb0 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5638178c0cb0 0x5638178c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f741d3a5640 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5638178c0cb0 0x5638178c30a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b759ba016a215232 server_cookie=d5a7df0029a7234b in_seq=0 out_seq=0 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.451+0000 7f74017fa640 10 client.17106 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74017fa640 1 -- 172.21.15.129:0/1477052887 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563817818b90 con 0x5638178c0cb0 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74017fa640 10 client.17106 handle_command_reply: tid=0 2024-09-05T01:32:00.475 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 client.17106 shutdown 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 msgr2=0x5638178ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f741401caa0 tx=0x7f741401cad0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 shutdown_connections 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5638178c0cb0 0x5638178c30a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178b1240 0x5638178aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6980 0x5638178ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 --2- 172.21.15.129:0/1477052887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178ae0e0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74017fa640 0 client.17106 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 >> 172.21.15.129:0/1477052887 conn(0x5638178a25f0 msgr2=0x5638178a82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.452+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 shutdown_connections 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.453+0000 7f74221e3740 1 -- 172.21.15.129:0/1477052887 wait complete. 2024-09-05T01:32:00.476 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.453+0000 7f74221e3740 20 client.17106 trim_cache size 0 max 16384 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 Processor -- start 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 -- start start 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178b1240 0x5638178c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f74181af500 con 0x5638178a6060 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74181acdd0 con 0x5638178a6980 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f74221e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74181aac20 con 0x5638178b1240 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f741dba6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45824/0 (socket says 172.21.15.129:45824) 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.470+0000 7f741dba6640 1 -- 172.21.15.129:0/53012330 learned_addr learned my addr 172.21.15.129:0/53012330 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:00.481 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741d3a5640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741e3a7640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178b1240 0x5638178c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 -- 172.21.15.129:0/53012330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178b1240 msgr2=0x5638178c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178b1240 0x5638178c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 -- 172.21.15.129:0/53012330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 msgr2=0x5638178bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 -- 172.21.15.129:0/53012330 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638178a5de0 con 0x5638178a6060 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741d3a5640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f741dba6640 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f741401cbc0 tx=0x7f74140275d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877029920bb410ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f74037fe640 1 -- 172.21.15.129:0/53012330 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741402b070 con 0x5638178a6060 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f74037fe640 1 -- 172.21.15.129:0/53012330 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741401c440 con 0x5638178a6060 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.471+0000 7f74037fe640 1 -- 172.21.15.129:0/53012330 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414021040 con 0x5638178a6060 2024-09-05T01:32:00.482 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- 172.21.15.129:0/53012330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 msgr2=0x5638178bc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f741401cbc0 tx=0x7f74140275d0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- 172.21.15.129:0/53012330 shutdown_connections 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178b1240 0x5638178c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 --2- 172.21.15.129:0/53012330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178a6060 0x5638178bc770 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- 172.21.15.129:0/53012330 >> 172.21.15.129:0/53012330 conn(0x5638178a25f0 msgr2=0x5638178bc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- 172.21.15.129:0/53012330 shutdown_connections 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- 172.21.15.129:0/53012330 wait complete. 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 -- start start 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.472+0000 7f74221e3740 1 Processor -- start 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.483 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ab290 con 0x5638178b1240 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ac700 con 0x5638178a6980 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ab960 con 0x5638178a6060 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74221e3740 10 client.0 _ll_register_callbacks cb 0x7f74100143d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741e3a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:57324/0 (socket says 172.21.15.129:57324) 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741e3a7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:45832/0 (socket says 172.21.15.129:45832) 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 -- 172.21.15.129:0/239412034 learned_addr learned my addr 172.21.15.129:0/239412034 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741d3a5640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 -- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 msgr2=0x5638178b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.484 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 -- 172.21.15.129:0/239412034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 msgr2=0x5638178b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 -- 172.21.15.129:0/239412034 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56381792a810 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741e3a7640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f741dba6640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7414005840 tx=0x7f7414005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c432959f1932f6b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.473+0000 7f74017fa640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414021030 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7414014840 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f741401c440 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7414049050 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 10 client.26518 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56381792bb30 con 0x5638178a6060 2024-09-05T01:32:00.485 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f741401c440 con 0x5638178a6060 2024-09-05T01:32:00.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f7414026460 con 0x5638178a6060 2024-09-05T01:32:00.486 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 10 client.26518 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 10 client.26518 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 20 client.26518 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56381792c1e0 0x56381792e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 4 client.26518 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "efbee404", "release": true, "await": true}] 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "efbee404", "release": true, "await": true}) -- 0x56381789d950 con 0x56381792c1e0 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.474+0000 7f741d3a5640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56381792c1e0 0x56381792e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.475+0000 7f741d3a5640 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56381792c1e0 0x56381792e5d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4c8758ef61d82ad server_cookie=4382bb7169b64d82 in_seq=0 out_seq=0 2024-09-05T01:32:00.487 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.475+0000 7f74017fa640 10 client.26518 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:00 smithi194 ceph-mon[34941]: pgmap v853: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 231 KiB/s wr, 38 op/s 2024-09-05T01:32:00.559 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74017fa640 1 -- 172.21.15.129:0/239412034 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56381789d950 con 0x56381792c1e0 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74017fa640 10 client.26518 handle_command_reply: tid=0 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 client.26518 shutdown 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 msgr2=0x5638178b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7414005840 tx=0x7f7414005b20 comp rx=0 tx=0).stop 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 shutdown_connections 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x56381792c1e0 0x56381792e5d0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5638178b1240 0x5638178b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5638178a6980 0x5638178b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.560 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 --2- 172.21.15.129:0/239412034 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5638178a6060 0x5638178b50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 >> 172.21.15.129:0/239412034 conn(0x5638178a25f0 msgr2=0x5638178bc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.558+0000 7f74017fa640 0 client.26518 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.559+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 shutdown_connections 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.559+0000 7f74221e3740 1 -- 172.21.15.129:0/239412034 wait complete. 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.559+0000 7f74221e3740 20 client.26518 trim_cache size 0 max 16384 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:00.561 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 64, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "efbee404": { 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 64, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:32:00.562 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:00.563 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:00.564 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:00.564 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 msgr2=0x7f74040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 0x7f74040a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f74100148c0 tx=0x7f7410001df0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 msgr2=0x7f7418198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:00.565 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f7408004270 comp rx=0 tx=0).stop 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 shutdown_connections 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f74040a2b80 0x7f74040a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7418106f10 0x7f741819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7418102790 0x7f7418198960 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 --2- 172.21.15.129:0/1172698721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7418101de0 0x7f7418198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 >> 172.21.15.129:0/1172698721 conn(0x7f74180fd8c0 msgr2=0x7f74180fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 shutdown_connections 2024-09-05T01:32:00.566 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:00.560+0000 7f741fe0a640 1 -- 172.21.15.129:0/1172698721 wait complete. 2024-09-05T01:32:01.307 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'efbee404', total seconds elapsed: 33.5 2024-09-05T01:32:01.307 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T01:32:01.308 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:32:01.743 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:02.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:02 smithi129 ceph-mon[30979]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 188 KiB/s wr, 30 op/s 2024-09-05T01:32:02.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 -- 172.21.15.129:0/2595705816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 msgr2=0x7f5788103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:02.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 --2- 172.21.15.129:0/2595705816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 0x7f5788103110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5784009900 tx=0x7f578402a490 comp rx=0 tx=0).stop 2024-09-05T01:32:02.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 -- 172.21.15.129:0/2595705816 shutdown_connections 2024-09-05T01:32:02.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 --2- 172.21.15.129:0/2595705816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f578810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 --2- 172.21.15.129:0/2595705816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f578810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 --2- 172.21.15.129:0/2595705816 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 0x7f5788103110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.374+0000 7f578f836640 1 -- 172.21.15.129:0/2595705816 >> 172.21.15.129:0/2595705816 conn(0x7f57880fc790 msgr2=0x7f57880febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:02.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.375+0000 7f578f836640 1 -- 172.21.15.129:0/2595705816 shutdown_connections 2024-09-05T01:32:02.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.375+0000 7f578f836640 1 -- 172.21.15.129:0/2595705816 wait complete. 2024-09-05T01:32:02.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 Processor -- start 2024-09-05T01:32:02.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 -- start start 2024-09-05T01:32:02.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 0x7f5788116140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f5788116680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f5788111260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5788102310 con 0x7f5788103650 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5788102190 con 0x7f578810c160 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.376+0000 7f578f836640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f5788102490 con 0x7f5788102d30 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578ddd3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f5788111260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:02.380 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578ddd3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f5788111260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42766/0 (socket says 172.21.15.129:42766) 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578ddd3640 1 -- 172.21.15.129:0/459046749 learned_addr learned my addr 172.21.15.129:0/459046749 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f5788116680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 -- 172.21.15.129:0/459046749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 msgr2=0x7f5788116140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 0x7f5788116140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 -- 172.21.15.129:0/459046749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 msgr2=0x7f5788111260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f5788111260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.377+0000 7f578cdd1640 1 -- 172.21.15.129:0/459046749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5788111b20 con 0x7f5788103650 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.378+0000 7f578cdd1640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f5788116680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f577800e7a0 tx=0x7f577800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.379+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577800ccc0 con 0x7f5788103650 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.379+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57780108e0 con 0x7f5788103650 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.379+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778018780 con 0x7f5788103650 2024-09-05T01:32:02.381 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.379+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5788111e10 con 0x7f5788103650 2024-09-05T01:32:02.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.379+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578811c030 con 0x7f5788103650 2024-09-05T01:32:02.382 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.380+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5788112070 con 0x7f5788103650 2024-09-05T01:32:02.383 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.383+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f5778010a90 con 0x7f5788103650 2024-09-05T01:32:02.384 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.384+0000 7f577e7fc640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 0x7f57740a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:02.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.384+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f57780cfab0 con 0x7f5788103650 2024-09-05T01:32:02.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.384+0000 7f578d5d2640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 0x7f57740a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:02.385 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.385+0000 7f578d5d2640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 0x7f57740a5190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f57840096c0 tx=0x7f5784031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:02.389 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.388+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f577809c680 con 0x7f5788103650 2024-09-05T01:32:02.410 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:02 smithi183 ceph-mon[34589]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 188 KiB/s wr, 30 op/s 2024-09-05T01:32:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:02 smithi194 ceph-mon[34941]: pgmap v854: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 188 KiB/s wr, 30 op/s 2024-09-05T01:32:02.775 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.774+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f578806b070 con 0x7f5788103650 2024-09-05T01:32:02.777 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.777+0000 7f577e7fc640 1 -- 172.21.15.129:0/459046749 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f577809bf80 con 0x7f5788103650 2024-09-05T01:32:02.778 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:02.778 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:32:02.778 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:32:02.781 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 msgr2=0x7f57740a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:02.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 0x7f57740a5190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f57840096c0 tx=0x7f5784031400 comp rx=0 tx=0).stop 2024-09-05T01:32:02.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 msgr2=0x7f5788116680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:02.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f5788116680 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f577800e7a0 tx=0x7f577800ec60 comp rx=0 tx=0).stop 2024-09-05T01:32:02.782 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 shutdown_connections 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f57740a2cd0 0x7f57740a5190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f578810c160 0x7f5788111260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5788103650 0x7f5788116680 secure :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f577800e7a0 tx=0x7f577800ec60 comp rx=0 tx=0).stop 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 --2- 172.21.15.129:0/459046749 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f5788102d30 0x7f5788116140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.781+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 >> 172.21.15.129:0/459046749 conn(0x7f57880fc790 msgr2=0x7f57880fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.782+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 shutdown_connections 2024-09-05T01:32:02.783 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:02.782+0000 7f578f836640 1 -- 172.21.15.129:0/459046749 wait complete. 2024-09-05T01:32:03.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:03 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/459046749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:03.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:03 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/459046749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:03.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:03 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/459046749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:03.676 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T01:32:03.677 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T01:32:04.087 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:04.362 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:04 smithi129 ceph-mon[30979]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 939 KiB/s wr, 56 op/s 2024-09-05T01:32:04.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:04 smithi183 ceph-mon[34589]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 939 KiB/s wr, 56 op/s 2024-09-05T01:32:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:04 smithi194 ceph-mon[34941]: pgmap v855: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 939 KiB/s wr, 56 op/s 2024-09-05T01:32:04.668 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.666+0000 7fb6876f4640 1 -- 172.21.15.129:0/1706085139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 msgr2=0x7fb680114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.666+0000 7fb6876f4640 1 --2- 172.21.15.129:0/1706085139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 0x7fb680114f80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb67801acb0 tx=0x7fb67803b9b0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 -- 172.21.15.129:0/1706085139 shutdown_connections 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 --2- 172.21.15.129:0/1706085139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 0x7fb680114f80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 --2- 172.21.15.129:0/1706085139 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074eb0 0x7fb680112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 --2- 172.21.15.129:0/1706085139 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074590 0x7fb680074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 -- 172.21.15.129:0/1706085139 >> 172.21.15.129:0/1706085139 conn(0x7fb6800fd880 msgr2=0x7fb6800ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 -- 172.21.15.129:0/1706085139 shutdown_connections 2024-09-05T01:32:04.669 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.667+0000 7fb6876f4640 1 -- 172.21.15.129:0/1706085139 wait complete. 2024-09-05T01:32:04.670 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 Processor -- start 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 -- start start 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074590 0x7fb680112360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 0x7fb68010e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb680108790 con 0x7fb680112b90 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb680108610 con 0x7fb680074eb0 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.668+0000 7fb6876f4640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb680108910 con 0x7fb680074590 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42776/0 (socket says 172.21.15.129:42776) 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 -- 172.21.15.129:0/3463077476 learned_addr learned my addr 172.21.15.129:0/3463077476 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb685490640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074590 0x7fb680112360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 -- 172.21.15.129:0/3463077476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074590 msgr2=0x7fb680112360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.671 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074590 0x7fb680112360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 -- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 msgr2=0x7fb68010e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 0x7fb68010e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.669+0000 7fb684c8f640 1 -- 172.21.15.129:0/3463077476 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68010f0c0 con 0x7fb680074eb0 2024-09-05T01:32:04.672 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb684c8f640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00b450 tx=0x7fb67c00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb6767fc640 1 -- 172.21.15.129:0/3463077476 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c015020 con 0x7fb680074eb0 2024-09-05T01:32:04.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb6767fc640 1 -- 172.21.15.129:0/3463077476 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb67c011d40 con 0x7fb680074eb0 2024-09-05T01:32:04.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb6767fc640 1 -- 172.21.15.129:0/3463077476 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67c00f340 con 0x7fb680074eb0 2024-09-05T01:32:04.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6801c4970 con 0x7fb680074eb0 2024-09-05T01:32:04.673 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.670+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801c4dd0 con 0x7fb680074eb0 2024-09-05T01:32:04.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 Processor -- start 2024-09-05T01:32:04.680 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 -- start start 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f745400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f747d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a4f6a02a0 con 0x559a4f745940 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a4f647b10 con 0x559a4f73a760 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb689acd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a4f69c260 con 0x559a4f73b080 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb685c91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb685c91640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42786/0 (socket says 172.21.15.129:42786) 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb685c91640 1 -- 172.21.15.129:0/4131514390 learned_addr learned my addr 172.21.15.129:0/4131514390 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.678+0000 7fb685490640 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f745400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 -- 172.21.15.129:0/4131514390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 msgr2=0x559a4f745400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f745400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 -- 172.21.15.129:0/4131514390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 msgr2=0x559a4f747d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f747d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 -- 172.21.15.129:0/4131514390 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4f411b90 con 0x559a4f73a760 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb6767fc640 1 -- 172.21.15.129:0/3463077476 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb67c018020 con 0x7fb680074eb0 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb685c91640 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb67803f480 tx=0x7fb67803f4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f832d1d8401642cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb66b7fe640 1 -- 172.21.15.129:0/4131514390 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678011020 con 0x559a4f73a760 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb66b7fe640 1 -- 172.21.15.129:0/4131514390 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb678003e20 con 0x559a4f73a760 2024-09-05T01:32:04.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.679+0000 7fb66b7fe640 1 -- 172.21.15.129:0/4131514390 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb678044030 con 0x559a4f73a760 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb6767fc640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 0x7fb66c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb685490640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 0x7fb66c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 -- 172.21.15.129:0/4131514390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 msgr2=0x559a4f73ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb67803f480 tx=0x7fb67803f4b0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb6767fc640 1 -- 172.21.15.129:0/3463077476 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb67c0ceea0 con 0x7fb680074eb0 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 -- 172.21.15.129:0/4131514390 shutdown_connections 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f747d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f745400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 --2- 172.21.15.129:0/4131514390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f73ab40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.680+0000 7fb689acd740 1 -- 172.21.15.129:0/4131514390 >> 172.21.15.129:0/4131514390 conn(0x559a4f736cf0 msgr2=0x559a4f7370d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:04.684 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.681+0000 7fb689acd740 1 -- 172.21.15.129:0/4131514390 shutdown_connections 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.681+0000 7fb689acd740 1 -- 172.21.15.129:0/4131514390 wait complete. 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.681+0000 7fb689acd740 1 -- start start 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 Processor -- start 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb685490640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 0x7fb66c0a5140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x559a4f737b50 tx=0x7fb6700090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f742870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f742db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f7432f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559a4f759520 con 0x559a4f745940 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a4f6286e0 con 0x559a4f73a760 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x559a4f7596a0 con 0x559a4f73b080 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb689acd740 10 client.0 _ll_register_callbacks cb 0x559a4f64ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.682+0000 7fb685490640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f742870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685490640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f742870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:42798/0 (socket says 172.21.15.129:42798) 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685490640 1 -- 172.21.15.129:0/1641801904 learned_addr learned my addr 172.21.15.129:0/1641801904 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f7432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 -- 172.21.15.129:0/1641801904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 msgr2=0x559a4f742db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f742db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 -- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 msgr2=0x559a4f742870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f742870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 -- 172.21.15.129:0/1641801904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4f762b10 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb685c91640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f7432f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb678003ba0 tx=0x7fb678003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f769f488bd1869 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.683+0000 7fb668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67804d070 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb678003e20 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb678040610 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb678059050 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb689acd740 10 client.17124 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.684+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a4f7654a0 con 0x559a4f745940 2024-09-05T01:32:04.687 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb678003e20 con 0x559a4f745940 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb678004270 con 0x559a4f745940 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 10 client.17124 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 10 client.17124 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 20 client.17124 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f766a80 0x559a4f768ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 4 client.17124 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.685+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a4f6ad9d0 con 0x559a4f766a80 2024-09-05T01:32:04.688 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.687+0000 7fb685490640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f766a80 0x559a4f768ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.688+0000 7fb685490640 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f766a80 0x559a4f768ec0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550dda14ac9b26ea server_cookie=61fa1520cb1b69a3 in_seq=0 out_seq=0 2024-09-05T01:32:04.689 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.688+0000 7fb668ff9640 10 client.17124 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:04.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.690+0000 7fb668ff9640 1 -- 172.21.15.129:0/1641801904 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a4f6ad9d0 con 0x559a4f766a80 2024-09-05T01:32:04.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.690+0000 7fb668ff9640 10 client.17124 handle_command_reply: tid=0 2024-09-05T01:32:04.691 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.690+0000 7fb689acd740 1 client.17124 shutdown 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.690+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 msgr2=0x559a4f7432f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.690+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f7432f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb678003ba0 tx=0x7fb678003bd0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 shutdown_connections 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f766a80 0x559a4f768ec0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f7432f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73b080 0x559a4f742db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 --2- 172.21.15.129:0/1641801904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73a760 0x559a4f742870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 >> 172.21.15.129:0/1641801904 conn(0x559a4f736cf0 msgr2=0x559a4f73c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:04.692 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb668ff9640 0 client.17124 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 shutdown_connections 2024-09-05T01:32:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.691+0000 7fb689acd740 1 -- 172.21.15.129:0/1641801904 wait complete. 2024-09-05T01:32:04.693 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.692+0000 7fb689acd740 20 client.17124 trim_cache size 0 max 16384 2024-09-05T01:32:04.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 Processor -- start 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 -- start start 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73b080 0x559a4f76e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f76ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801c8e10 con 0x559a4f745940 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801c66e0 con 0x559a4f73b080 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.721+0000 7fb689acd740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb680073920 con 0x559a4f73a760 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:35738/0 (socket says 172.21.15.129:35738) 2024-09-05T01:32:04.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 -- 172.21.15.129:0/827904445 learned_addr learned my addr 172.21.15.129:0/827904445 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685c91640 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f76ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 -- 172.21.15.129:0/827904445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73b080 msgr2=0x559a4f76e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73b080 0x559a4f76e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 -- 172.21.15.129:0/827904445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 msgr2=0x559a4f76ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f76ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 -- 172.21.15.129:0/827904445 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4f73aeb0 con 0x559a4f73a760 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.722+0000 7fb685490640 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb670009930 tx=0x7fb670018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab9b1c45fa59f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.723+0000 7fb66affd640 1 -- 172.21.15.129:0/827904445 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670024b30 con 0x559a4f73a760 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.723+0000 7fb66affd640 1 -- 172.21.15.129:0/827904445 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb670030070 con 0x559a4f73a760 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.723+0000 7fb66affd640 1 -- 172.21.15.129:0/827904445 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6700223e0 con 0x559a4f73a760 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 -- 172.21.15.129:0/827904445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 msgr2=0x559a4f743140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb670009930 tx=0x7fb670018000 comp rx=0 tx=0).stop 2024-09-05T01:32:04.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 -- 172.21.15.129:0/827904445 shutdown_connections 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f745940 0x559a4f76ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f73b080 0x559a4f76e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 --2- 172.21.15.129:0/827904445 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f743140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.724+0000 7fb689acd740 1 -- 172.21.15.129:0/827904445 >> 172.21.15.129:0/827904445 conn(0x559a4f736cf0 msgr2=0x559a4f7448f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.725+0000 7fb689acd740 1 -- 172.21.15.129:0/827904445 shutdown_connections 2024-09-05T01:32:04.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.725+0000 7fb689acd740 1 -- 172.21.15.129:0/827904445 wait complete. 2024-09-05T01:32:04.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.725+0000 7fb689acd740 1 -- start start 2024-09-05T01:32:04.729 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.726+0000 7fb689acd740 1 Processor -- start 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.726+0000 7fb689acd740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f73b080 0x559a4f73f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f745940 0x559a4f73f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:35746/0 (socket says 172.21.15.129:35746) 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 -- 172.21.15.129:0/2909565350 learned_addr learned my addr 172.21.15.129:0/2909565350 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb680073f90 con 0x559a4f73b080 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801c6010 con 0x559a4f745940 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb6801c5270 con 0x559a4f73a760 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb689acd740 10 client.0 _ll_register_callbacks cb 0x7fb67803ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685c91640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f745940 0x559a4f73f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.730 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 -- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f745940 msgr2=0x559a4f73f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f745940 0x559a4f73f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 -- 172.21.15.129:0/2909565350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f73b080 msgr2=0x559a4f73f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f73b080 0x559a4f73f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.727+0000 7fb685490640 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4f740020 con 0x559a4f73a760 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb685490640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb670006dc0 tx=0x7fb670024d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1070f9ea108b4e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb668ff9640 10 client.0 ms_handle_connect on v2:172.21.15.194:3300/0 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb670005da0 con 0x559a4f73a760 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mon.1 v2:172.21.15.194:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb670014440 con 0x559a4f73a760 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb670030070 con 0x559a4f73a760 2024-09-05T01:32:04.731 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.728+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb670046050 con 0x559a4f73a760 2024-09-05T01:32:04.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 10 client.26533 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:04.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a4f7d1620 con 0x559a4f73a760 2024-09-05T01:32:04.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mon.1 v2:172.21.15.194:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb67001b040 con 0x559a4f73a760 2024-09-05T01:32:04.732 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mon.1 v2:172.21.15.194:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb670020040 con 0x559a4f73a760 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 10 client.26533 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 10 client.26533 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 20 client.26533 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f7d1c50 0x559a4f7d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 4 client.26533 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.731+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fb670030070 con 0x559a4f7d1c50 2024-09-05T01:32:04.733 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.732+0000 7fb684c8f640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f7d1c50 0x559a4f7d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:04.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.733+0000 7fb684c8f640 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f7d1c50 0x559a4f7d4040 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8c990c0af59b797 server_cookie=593c965b4c75ab15 in_seq=0 out_seq=0 2024-09-05T01:32:04.734 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:04.733+0000 7fb668ff9640 10 client.26533 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:05.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb668ff9640 1 -- 172.21.15.129:0/2909565350 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fb670030070 con 0x559a4f7d1c50 2024-09-05T01:32:05.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb668ff9640 10 client.26533 handle_command_reply: tid=0 2024-09-05T01:32:05.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 client.26533 shutdown 2024-09-05T01:32:05.026 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 msgr2=0x559a4f75efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb670006dc0 tx=0x7fb670024d60 comp rx=0 tx=0).stop 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 shutdown_connections 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x559a4f7d1c50 0x559a4f7d4040 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a4f745940 0x559a4f73f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559a4f73b080 0x559a4f73f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb668ff9640 0 client.26533 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 --2- 172.21.15.129:0/2909565350 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x559a4f73a760 0x559a4f75efb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 >> 172.21.15.129:0/2909565350 conn(0x559a4f736cf0 msgr2=0x559a4f735650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 shutdown_connections 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 1 -- 172.21.15.129:0/2909565350 wait complete. 2024-09-05T01:32:05.027 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.025+0000 7fb689acd740 20 client.26533 trim_cache size 0 max 16384 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 66, 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "599cc8ea": { 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 66, 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:05.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "QUIESCED", 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:05.029 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:05.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 msgr2=0x7fb66c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:05.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 0x7fb66c0a5140 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x559a4f737b50 tx=0x7fb6700090e0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 msgr2=0x7fb68010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:05.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00b450 tx=0x7fb67c00b910 comp rx=0 tx=0).stop 2024-09-05T01:32:05.030 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 shutdown_connections 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb66c0a2c80 0x7fb66c0a5140 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb680112b90 0x7fb68010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb680074eb0 0x7fb68010e400 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 --2- 172.21.15.129:0/3463077476 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb680074590 0x7fb680112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 >> 172.21.15.129:0/3463077476 conn(0x7fb6800fd880 msgr2=0x7fb6800ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 shutdown_connections 2024-09-05T01:32:05.031 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:05.026+0000 7fb6876f4640 1 -- 172.21.15.129:0/3463077476 wait complete. 2024-09-05T01:32:05.206 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 900.0 s, 829.0 tps, lat 807.312 ms stddev 4500.883 2024-09-05T01:32:05.739 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '599cc8ea', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-05T01:32:06.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:06 smithi183 ceph-mon[34589]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-09-05T01:32:06.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:06 smithi129 ceph-mon[30979]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-09-05T01:32:06.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:06 smithi194 ceph-mon[34941]: pgmap v856: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-09-05T01:32:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:08 smithi183 ceph-mon[34589]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-09-05T01:32:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:08 smithi129 ceph-mon[30979]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-09-05T01:32:08.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:08 smithi194 ceph-mon[34941]: pgmap v857: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 -- 172.21.15.129:0/956133098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84102d50 msgr2=0x7fec841031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 --2- 172.21.15.129:0/956133098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84102d50 0x7fec841031d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fec80009900 tx=0x7fec8002a410 comp rx=0 tx=0).stop 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 -- 172.21.15.129:0/956133098 shutdown_connections 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 --2- 172.21.15.129:0/956133098 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec84103710 0x7fec84109fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 --2- 172.21.15.129:0/956133098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84102d50 0x7fec841031d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 --2- 172.21.15.129:0/956133098 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84101b60 0x7fec84101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 -- 172.21.15.129:0/956133098 >> 172.21.15.129:0/956133098 conn(0x7fec840fd2e0 msgr2=0x7fec840ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 -- 172.21.15.129:0/956133098 shutdown_connections 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.948+0000 7fec8be53640 1 -- 172.21.15.129:0/956133098 wait complete. 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 Processor -- start 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 -- start start 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84102d50 0x7fec84198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec84103710 0x7fec8419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410d690 con 0x7fec84101b60 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410d510 con 0x7fec84102d50 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec8be53640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fec8410d810 con 0x7fec84103710 2024-09-05T01:32:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55510/0 (socket says 172.21.15.129:55510) 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 -- 172.21.15.129:0/42030688 learned_addr learned my addr 172.21.15.129:0/42030688 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec893ee640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84102d50 0x7fec84198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 -- 172.21.15.129:0/42030688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec84103710 msgr2=0x7fec8419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec84103710 0x7fec8419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 -- 172.21.15.129:0/42030688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84102d50 msgr2=0x7fec84198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84102d50 0x7fec84198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.949+0000 7fec89bef640 1 -- 172.21.15.129:0/42030688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8419f910 con 0x7fec84101b60 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec89bef640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d770 tx=0x7fec7800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec76ffd640 1 -- 172.21.15.129:0/42030688 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78017070 con 0x7fec84101b60 2024-09-05T01:32:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec76ffd640 1 -- 172.21.15.129:0/42030688 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec78009c30 con 0x7fec84101b60 2024-09-05T01:32:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8419fc00 con 0x7fec84101b60 2024-09-05T01:32:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec76ffd640 1 -- 172.21.15.129:0/42030688 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78003d00 con 0x7fec84101b60 2024-09-05T01:32:08.954 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.950+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec841a0150 con 0x7fec84101b60 2024-09-05T01:32:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.952+0000 7fec76ffd640 1 -- 172.21.15.129:0/42030688 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fec78003ea0 con 0x7fec84101b60 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 Processor -- start 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 -- start start 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556db561a2f0 con 0x556db56b72e0 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556db55c2270 con 0x556db56bc370 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556db56162b0 con 0x556db56b69a0 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec893ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55526/0 (socket says 172.21.15.129:55526) 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.953+0000 7fec89bef640 1 -- 172.21.15.129:0/2650215533 learned_addr learned my addr 172.21.15.129:0/2650215533 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec76ffd640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 0x7fec700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.959 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 0x7fec700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 -- 172.21.15.129:0/2650215533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 msgr2=0x556db56b6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56b6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 -- 172.21.15.129:0/2650215533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 msgr2=0x556db56be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 -- 172.21.15.129:0/2650215533 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db55c1340 con 0x556db56b72e0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec893ee640 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec76ffd640 1 -- 172.21.15.129:0/42030688 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec780cf240 con 0x7fec84101b60 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec89bef640 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fec78014e90 tx=0x7fec78023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f138c6a3c82e0e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 0x7fec700a50f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009f50 tx=0x7fec7c005e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec6ffff640 1 -- 172.21.15.129:0/2650215533 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7809b170 con 0x556db56b72e0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec6ffff640 1 -- 172.21.15.129:0/2650215533 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec78014ec0 con 0x556db56b72e0 2024-09-05T01:32:08.960 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.954+0000 7fec6ffff640 1 -- 172.21.15.129:0/2650215533 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780d2d70 con 0x556db56b72e0 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2650215533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 msgr2=0x556db56bbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fec78014e90 tx=0x7fec78023000 comp rx=0 tx=0).stop 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2650215533 shutdown_connections 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56bbda0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2650215533 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56b6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2650215533 >> 172.21.15.129:0/2650215533 conn(0x556db56b2010 msgr2=0x556db56b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2650215533 shutdown_connections 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2650215533 wait complete. 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 -- start start 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.955+0000 7fec8ebf0740 1 Processor -- start 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56c0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.961 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556db56bf280 con 0x556db56b72e0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556db55a3290 con 0x556db56bc370 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556db56bf400 con 0x556db56b69a0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8ebf0740 10 client.0 _ll_register_callbacks cb 0x556db55d4140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec89bef640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56c0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55528/0 (socket says 172.21.15.129:55528) 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 -- 172.21.15.129:0/927564231 learned_addr learned my addr 172.21.15.129:0/927564231 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 -- 172.21.15.129:0/927564231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 msgr2=0x556db56c0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56c0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 -- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 msgr2=0x556db56c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec893ee640 1 -- 172.21.15.129:0/927564231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556db532d210 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.956+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec893ee640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fec80009900 tx=0x7fec800317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c440b9edd9dda0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec80035040 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec80004450 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec78014ec0 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fec8004e050 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 10 client.17139 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556db53f9620 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec78014ec0 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fec8003c640 con 0x556db56b72e0 2024-09-05T01:32:08.963 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 10 client.17139 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 20 client.17139 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db56cc790 0x556db56cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 4 client.17139 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.957+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556db5627a20 con 0x556db56cc790 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.958+0000 7fec89bef640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db56cc790 0x556db56cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.958+0000 7fec6d7fa640 10 client.17139 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:08.964 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.958+0000 7fec89bef640 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db56cc790 0x556db56cebd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3b9bf8c80e5253 server_cookie=69a645249f735eaa in_seq=0 out_seq=0 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec6d7fa640 1 -- 172.21.15.129:0/927564231 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556db5627a20 con 0x556db56cc790 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec6d7fa640 10 client.17139 handle_command_reply: tid=0 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 client.17139 shutdown 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 msgr2=0x556db56c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fec80009900 tx=0x7fec800317b0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 shutdown_connections 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db56cc790 0x556db56cebd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.965 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56bc370 0x556db56c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b72e0 0x556db56c1010 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/927564231 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56b69a0 0x556db56c0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec6d7fa640 0 client.17139 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 >> 172.21.15.129:0/927564231 conn(0x556db56b2010 msgr2=0x556db56b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 shutdown_connections 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 1 -- 172.21.15.129:0/927564231 wait complete. 2024-09-05T01:32:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.959+0000 7fec8ebf0740 20 client.17139 trim_cache size 0 max 16384 2024-09-05T01:32:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 Processor -- start 2024-09-05T01:32:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 -- start start 2024-09-05T01:32:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db56b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.978 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db56b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec841ad0a0 con 0x556db56b69a0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec841aa970 con 0x556db56b72e0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.973+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fec841a87c0 con 0x556db56bc370 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55538/0 (socket says 172.21.15.129:55538) 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 -- 172.21.15.129:0/2239543530 learned_addr learned my addr 172.21.15.129:0/2239543530 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec893ee640 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db56b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db56b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 -- 172.21.15.129:0/2239543530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 msgr2=0x556db56b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db56b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 -- 172.21.15.129:0/2239543530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 msgr2=0x556db56b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db56b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 -- 172.21.15.129:0/2239543530 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db56cb0a0 con 0x556db56b69a0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec89bef640 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fec78004610 tx=0x7fec780d2c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73497683eb1bb5b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec6f7fe640 1 -- 172.21.15.129:0/2239543530 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78023c00 con 0x556db56b69a0 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec6f7fe640 1 -- 172.21.15.129:0/2239543530 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec78009de0 con 0x556db56b69a0 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.974+0000 7fec6f7fe640 1 -- 172.21.15.129:0/2239543530 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec78099770 con 0x556db56b69a0 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2239543530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 msgr2=0x556db56b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fec78004610 tx=0x7fec780d2c80 comp rx=0 tx=0).stop 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2239543530 shutdown_connections 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db56b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db56b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2239543530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db56b6d80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2239543530 >> 172.21.15.129:0/2239543530 conn(0x556db56b2010 msgr2=0x556db56b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2239543530 shutdown_connections 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2239543530 wait complete. 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- start start 2024-09-05T01:32:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 Processor -- start 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db5732c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db5739cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fec841a8e30 con 0x556db56b69a0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec841aa2a0 con 0x556db56b72e0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.975+0000 7fec8ebf0740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fec841a9500 con 0x556db56bc370 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec8ebf0740 10 client.0 _ll_register_callbacks cb 0x7fec80031d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55554/0 (socket says 172.21.15.129:55554) 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 -- 172.21.15.129:0/2622876895 learned_addr learned my addr 172.21.15.129:0/2622876895 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec893ee640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db5732c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec8a3f0640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db5739cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 -- 172.21.15.129:0/2622876895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 msgr2=0x556db5739cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.981 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db5739cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 -- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 msgr2=0x556db5732c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db5732c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 -- 172.21.15.129:0/2622876895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556db573a2c0 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec89bef640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fec780a3d70 tx=0x7fec78097720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633a6f4f15db47dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec780b4060 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fec780af790 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec78009de0 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fec78020020 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec8ebf0740 10 client.17145 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.976+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556db573b5d0 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec78009de0 con 0x556db56b69a0 2024-09-05T01:32:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fec780cc950 con 0x556db56b69a0 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 10 client.17145 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 20 client.17145 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db573bb40 0x556db573df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 4 client.17145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556db5621d70 con 0x556db573bb40 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec893ee640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db573bb40 0x556db573df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec893ee640 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db573bb40 0x556db573df30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c94083b43029bbb server_cookie=e31c20470aa9649e in_seq=0 out_seq=0 2024-09-05T01:32:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.977+0000 7fec6d7fa640 10 client.17145 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.995+0000 7fec6d7fa640 1 -- 172.21.15.129:0/2622876895 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556db5621d70 con 0x556db573bb40 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec6d7fa640 10 client.17145 handle_command_reply: tid=0 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 client.17145 shutdown 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 msgr2=0x556db5732730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fec780a3d70 tx=0x7fec78097720 comp rx=0 tx=0).stop 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 shutdown_connections 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556db573bb40 0x556db573df30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556db56bc370 0x556db5739cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec6d7fa640 0 client.17145 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556db56b72e0 0x556db5732c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 --2- 172.21.15.129:0/2622876895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556db56b69a0 0x556db5732730 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:08.997 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 >> 172.21.15.129:0/2622876895 conn(0x556db56b2010 msgr2=0x556db56b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 shutdown_connections 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 1 -- 172.21.15.129:0/2622876895 wait complete. 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.996+0000 7fec8ebf0740 20 client.17145 trim_cache size 0 max 16384 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:32:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:32:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 msgr2=0x7fec700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 0x7fec700a50f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009f50 tx=0x7fec7c005e00 comp rx=0 tx=0).stop 2024-09-05T01:32:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 msgr2=0x7fec84197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:08.999 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d770 tx=0x7fec7800dc30 comp rx=0 tx=0).stop 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 shutdown_connections 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fec700a2c30 0x7fec700a50f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fec84103710 0x7fec8419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec84102d50 0x7fec84198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 --2- 172.21.15.129:0/42030688 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fec84101b60 0x7fec84197d00 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 >> 172.21.15.129:0/42030688 conn(0x7fec840fd2e0 msgr2=0x7fec84106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 shutdown_connections 2024-09-05T01:32:09.000 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:08.997+0000 7fec8be53640 1 -- 172.21.15.129:0/42030688 wait complete. 2024-09-05T01:32:10.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 905.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:10.302 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:10 smithi194 ceph-mon[34941]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-05T01:32:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:10 smithi183 ceph-mon[34589]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-05T01:32:10.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:10 smithi129 ceph-mon[30979]: pgmap v858: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 3.6 MiB/s wr, 262 op/s 2024-09-05T01:32:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:12 smithi183 ceph-mon[34589]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-05T01:32:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:12 smithi129 ceph-mon[30979]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-05T01:32:12.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:12 smithi194 ceph-mon[34941]: pgmap v859: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-09-05T01:32:14.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:14 smithi183 ceph-mon[34589]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-09-05T01:32:14.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:14 smithi129 ceph-mon[30979]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-09-05T01:32:14.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:14 smithi194 ceph-mon[34941]: pgmap v860: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.8 MiB/s wr, 271 op/s 2024-09-05T01:32:15.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 910.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:16.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:16 smithi183 ceph-mon[34589]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-05T01:32:16.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:16 smithi129 ceph-mon[30979]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-05T01:32:16.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:16 smithi194 ceph-mon[34941]: pgmap v861: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.1 MiB/s wr, 250 op/s 2024-09-05T01:32:18.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:18 smithi183 ceph-mon[34589]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-05T01:32:18.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:18 smithi129 ceph-mon[30979]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-05T01:32:18.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:18 smithi194 ceph-mon[34941]: pgmap v862: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.8 MiB/s wr, 207 op/s 2024-09-05T01:32:20.212 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 915.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:20.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:20 smithi183 ceph-mon[34589]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T01:32:20.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:20 smithi129 ceph-mon[30979]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T01:32:20.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:20 smithi194 ceph-mon[34941]: pgmap v863: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-05T01:32:22.097 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- 172.21.15.129:0/1445510378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 msgr2=0x7f409c109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 --2- 172.21.15.129:0/1445510378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c109cc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4094009ca0 tx=0x7f409402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- 172.21.15.129:0/1445510378 shutdown_connections 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 --2- 172.21.15.129:0/1445510378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c109cc0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 --2- 172.21.15.129:0/1445510378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 0x7f409c107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 --2- 172.21.15.129:0/1445510378 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 0x7f409c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- 172.21.15.129:0/1445510378 >> 172.21.15.129:0/1445510378 conn(0x7f409c0fc1b0 msgr2=0x7f409c0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- 172.21.15.129:0/1445510378 shutdown_connections 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- 172.21.15.129:0/1445510378 wait complete. 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 Processor -- start 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.095+0000 7f40a21bd640 1 -- start start 2024-09-05T01:32:22.098 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 0x7f409c197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 0x7f409c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d6a0 con 0x7f409c1078d0 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d520 con 0x7f409c104fa0 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f40a21bd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f409c10d820 con 0x7f409c102670 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37020/0 (socket says 172.21.15.129:37020) 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 -- 172.21.15.129:0/2420058721 learned_addr learned my addr 172.21.15.129:0/2420058721 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409b7fe640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 0x7f409c197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409affd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 0x7f409c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 -- 172.21.15.129:0/2420058721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 msgr2=0x7f409c197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.099 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 0x7f409c197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 -- 172.21.15.129:0/2420058721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 msgr2=0x7f409c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 0x7f409c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 -- 172.21.15.129:0/2420058721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c19f950 con 0x7f409c1078d0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f409bfff640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4094031040 tx=0x7f4094003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.096+0000 7f4098ff9640 1 -- 172.21.15.129:0/2420058721 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094004040 con 0x7f409c1078d0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.097+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c19fbe0 con 0x7f409c1078d0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.097+0000 7f4098ff9640 1 -- 172.21.15.129:0/2420058721 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4094004690 con 0x7f409c1078d0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.097+0000 7f4098ff9640 1 -- 172.21.15.129:0/2420058721 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094030bd0 con 0x7f409c1078d0 2024-09-05T01:32:22.100 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.097+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a0130 con 0x7f409c1078d0 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f4098ff9640 1 -- 172.21.15.129:0/2420058721 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f40940041e0 con 0x7f409c1078d0 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 Processor -- start 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 -- start start 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b61da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.107 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1ac02f0 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1a68270 con 0x55b4e1b62370 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f40a5034740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1abc2b0 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f4098ff9640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 0x7f40840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f409affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.099+0000 7f409affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37028/0 (socket says 172.21.15.129:37028) 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 -- 172.21.15.129:0/3525032030 learned_addr learned my addr 172.21.15.129:0/3525032030 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409bfff640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 0x7f40840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b61da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f4098ff9640 1 -- 172.21.15.129:0/2420058721 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f40940f0ad0 con 0x7f409c1078d0 2024-09-05T01:32:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 -- 172.21.15.129:0/3525032030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5d2e0 msgr2=0x55b4e1b61da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b61da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 -- 172.21.15.129:0/3525032030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 msgr2=0x55b4e1b64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b64830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 -- 172.21.15.129:0/3525032030 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e1a67340 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409affd640 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4090005870 tx=0x7f4090009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707a495716e273c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f409bfff640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 0x7f40840a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f40940bceb0 tx=0x7f40940095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f4081ffb640 1 -- 172.21.15.129:0/3525032030 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090004030 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f4081ffb640 1 -- 172.21.15.129:0/3525032030 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40900035f0 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.100+0000 7f4081ffb640 1 -- 172.21.15.129:0/3525032030 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090010690 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- 172.21.15.129:0/3525032030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 msgr2=0x55b4e1b5cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4090005870 tx=0x7f4090009850 comp rx=0 tx=0).stop 2024-09-05T01:32:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- 172.21.15.129:0/3525032030 shutdown_connections 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b61da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 --2- 172.21.15.129:0/3525032030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b5cda0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- 172.21.15.129:0/3525032030 >> 172.21.15.129:0/3525032030 conn(0x55b4e1b58010 msgr2=0x55b4e1b58410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- 172.21.15.129:0/3525032030 shutdown_connections 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- 172.21.15.129:0/3525032030 wait complete. 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.101+0000 7f40a5034740 1 -- start start 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 Processor -- start 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b66b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b62370 0x55b4e1b6e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1b65340 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1a49290 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55b4e1b654c0 con 0x55b4e1b62370 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f40a5034740 10 client.0 _ll_register_callbacks cb 0x55b4e1a7a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37032/0 (socket says 172.21.15.129:37032) 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 -- 172.21.15.129:0/3144567811 learned_addr learned my addr 172.21.15.129:0/3144567811 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409affd640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b62370 0x55b4e1b6e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 -- 172.21.15.129:0/3144567811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b62370 msgr2=0x55b4e1b6e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b62370 0x55b4e1b6e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409affd640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b66b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 -- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 msgr2=0x55b4e1b66b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.111 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b66b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 -- 172.21.15.129:0/3144567811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e17d3210 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409affd640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b66b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f408c00bf70 tx=0x7f408c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128454e711b3562 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f407affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.102+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c021060 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f408c010690 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40900035f0 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f408c028050 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 10 client.17157 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b4e189f620 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40900035f0 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f408c00ea60 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.112 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 10 client.17157 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 20 client.17157 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1b728b0 0x55b4e1b74cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 4 client.17157 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e1acda20 con 0x55b4e1b728b0 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.103+0000 7f409affd640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1b728b0 0x55b4e1b74cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.104+0000 7f409affd640 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1b728b0 0x55b4e1b74cc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e7ec96391cc4b2 server_cookie=5ad543f5636da93c in_seq=0 out_seq=0 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.104+0000 7f407affd640 10 client.17157 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f407affd640 1 -- 172.21.15.129:0/3144567811 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4e1acda20 con 0x55b4e1b728b0 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f407affd640 10 client.17157 handle_command_reply: tid=0 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 client.17157 shutdown 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 msgr2=0x55b4e1b670d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f408c00bf70 tx=0x7f408c0128e0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 shutdown_connections 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1b728b0 0x55b4e1b74cc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b62370 0x55b4e1b6e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b670d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f407affd640 0 client.17157 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 --2- 172.21.15.129:0/3144567811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b66b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 >> 172.21.15.129:0/3144567811 conn(0x55b4e1b58010 msgr2=0x55b4e1b59820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 shutdown_connections 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 1 -- 172.21.15.129:0/3144567811 wait complete. 2024-09-05T01:32:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.105+0000 7f40a5034740 20 client.17157 trim_cache size 0 max 16384 2024-09-05T01:32:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 Processor -- start 2024-09-05T01:32:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 -- start start 2024-09-05T01:32:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b65890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ad080 con 0x55b4e1b62370 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aa950 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.119+0000 7f40a5034740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a87a0 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37818/0 (socket says 172.21.15.129:37818) 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 -- 172.21.15.129:0/1138582336 learned_addr learned my addr 172.21.15.129:0/1138582336 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b65890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409b7fe640 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 -- 172.21.15.129:0/1138582336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 msgr2=0x55b4e1b65890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b65890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 -- 172.21.15.129:0/1138582336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5d2e0 msgr2=0x55b4e1b6e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 -- 172.21.15.129:0/1138582336 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e1b75340 con 0x55b4e1b62370 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f409affd640 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55b4e1b6f5c0 tx=0x7f4090020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c459cc2d357365e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f40817fa640 1 -- 172.21.15.129:0/1138582336 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090003b20 con 0x55b4e1b62370 2024-09-05T01:32:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f40817fa640 1 -- 172.21.15.129:0/1138582336 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4090005590 con 0x55b4e1b62370 2024-09-05T01:32:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.120+0000 7f40817fa640 1 -- 172.21.15.129:0/1138582336 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409002cb00 con 0x55b4e1b62370 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- 172.21.15.129:0/1138582336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 msgr2=0x55b4e1b570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55b4e1b6f5c0 tx=0x7f4090020040 comp rx=0 tx=0).stop 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- 172.21.15.129:0/1138582336 shutdown_connections 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b62370 0x55b4e1b570b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.126 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 --2- 172.21.15.129:0/1138582336 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b65890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- 172.21.15.129:0/1138582336 >> 172.21.15.129:0/1138582336 conn(0x55b4e1b58010 msgr2=0x55b4e1b569a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- 172.21.15.129:0/1138582336 shutdown_connections 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- 172.21.15.129:0/1138582336 wait complete. 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 -- start start 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.121+0000 7f40a5034740 1 Processor -- start 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b6be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a8e10 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aa280 con 0x55b4e1b62370 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1a94e0 con 0x55b4e1b5c9a0 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 10 client.0 _ll_register_callbacks cb 0x7f408c00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.127 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37824/0 (socket says 172.21.15.129:37824) 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 -- 172.21.15.129:0/3231477016 learned_addr learned my addr 172.21.15.129:0/3231477016 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409affd640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409affd640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 -- 172.21.15.129:0/3231477016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 msgr2=0x55b4e1b6be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b6be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 -- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 msgr2=0x55b4e1b6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 -- 172.21.15.129:0/3231477016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e1b6d990 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409affd640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f409b7fe640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55b4e1b68070 tx=0x7f408c00bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c0616c717912fd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f407affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c000e60 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4090005590 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.128 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.122+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f408c0055d0 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f408c03a050 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 10 client.17163 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b4e1bddc20 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4090005590 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f408c00e5a0 con 0x55b4e1b5d2e0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 10 client.17163 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 20 client.17163 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1bde620 0x55b4e1be0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 4 client.17163 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4e1ac7d70 con 0x55b4e1bde620 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.123+0000 7f409affd640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1bde620 0x55b4e1be0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.124+0000 7f407affd640 10 client.17163 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:22.129 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.124+0000 7f409affd640 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1bde620 0x55b4e1be0a40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916ac90badc16087 server_cookie=f2ffb14e15a7f5f0 in_seq=0 out_seq=0 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f407affd640 1 -- 172.21.15.129:0/3231477016 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4e1ac7d70 con 0x55b4e1bde620 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f407affd640 10 client.17163 handle_command_reply: tid=0 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 client.17163 shutdown 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 msgr2=0x55b4e1b6c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55b4e1b68070 tx=0x7f408c00bc60 comp rx=0 tx=0).stop 2024-09-05T01:32:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 shutdown_connections 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55b4e1bde620 0x55b4e1be0a40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b4e1b62370 0x55b4e1b6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f407affd640 0 client.17163 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b4e1b5d2e0 0x55b4e1b6c370 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 --2- 172.21.15.129:0/3231477016 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55b4e1b5c9a0 0x55b4e1b6be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 >> 172.21.15.129:0/3231477016 conn(0x55b4e1b58010 msgr2=0x55b4e1b571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 shutdown_connections 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 1 -- 172.21.15.129:0/3231477016 wait complete. 2024-09-05T01:32:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.134+0000 7f40a5034740 20 client.17163 trim_cache size 0 max 16384 2024-09-05T01:32:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:32:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:32:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:32:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 msgr2=0x7f40840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 0x7f40840a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f40940bceb0 tx=0x7f40940095c0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 msgr2=0x7f409c19f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4094031040 tx=0x7f4094003ba0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 shutdown_connections 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f40840a2c30 0x7f40840a50f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f409c1078d0 0x7f409c19f2b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.138 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f409c104fa0 0x7f409c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 --2- 172.21.15.129:0/2420058721 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f409c102670 0x7f409c197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 >> 172.21.15.129:0/2420058721 conn(0x7f409c0fc1b0 msgr2=0x7f409c103050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 shutdown_connections 2024-09-05T01:32:22.139 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:22.135+0000 7f40a21bd640 1 -- 172.21.15.129:0/2420058721 wait complete. 2024-09-05T01:32:22.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:22 smithi183 ceph-mon[34589]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 503 KiB/s wr, 47 op/s 2024-09-05T01:32:22.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:22 smithi129 ceph-mon[30979]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 503 KiB/s wr, 47 op/s 2024-09-05T01:32:22.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:22 smithi194 ceph-mon[34941]: pgmap v864: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 503 KiB/s wr, 47 op/s 2024-09-05T01:32:24.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:24 smithi183 ceph-mon[34589]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 242 KiB/s wr, 41 op/s 2024-09-05T01:32:24.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:24 smithi129 ceph-mon[30979]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 242 KiB/s wr, 41 op/s 2024-09-05T01:32:24.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:24 smithi194 ceph-mon[34941]: pgmap v865: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 242 KiB/s wr, 41 op/s 2024-09-05T01:32:25.211 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 920.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:32:25.412 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:32:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:32:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:32:25.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:32:25.494 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:32:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:32:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:32:25.502 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:32:26.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:26 smithi183 ceph-mon[34589]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:32:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:32:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:32:26.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:32:26.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:32:26.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:32:26.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:32:26.493 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:26 smithi129 ceph-mon[30979]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:32:26.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: pgmap v866: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 219 KiB/s wr, 37 op/s 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi194 to 176.0M 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi183 to 184573849: error parsing value: Value '184573849' is below minimum 939524096 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi129 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T01:32:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:26 smithi194 ceph-mon[34941]: Unable to set osd_memory_target on smithi194 to 184596787: error parsing value: Value '184596787' is below minimum 939524096 2024-09-05T01:32:28.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:28 smithi183 ceph-mon[34589]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 KiB/s wr, 31 op/s 2024-09-05T01:32:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:28 smithi129 ceph-mon[30979]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 KiB/s wr, 31 op/s 2024-09-05T01:32:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:28 smithi194 ceph-mon[34941]: pgmap v867: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 204 KiB/s wr, 31 op/s 2024-09-05T01:32:30.210 INFO:teuthology.orchestra.run.smithi129.stderr:progress: 925.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-05T01:32:30.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:30 smithi183 ceph-mon[34589]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:32:30.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:30 smithi129 ceph-mon[30979]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:32:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:30 smithi194 ceph-mon[34941]: pgmap v868: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 194 KiB/s wr, 32 op/s 2024-09-05T01:32:31.340 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:32:31.599 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:32.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 -- 172.21.15.129:0/155175694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8106f10 msgr2=0x7f56d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 --2- 172.21.15.129:0/155175694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8106f10 0x7f56d810ea90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009900 tx=0x7f56c802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 -- 172.21.15.129:0/155175694 shutdown_connections 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 --2- 172.21.15.129:0/155175694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8106f10 0x7f56d810ea90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 --2- 172.21.15.129:0/155175694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8102790 0x7f56d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 --2- 172.21.15.129:0/155175694 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8101de0 0x7f56d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 -- 172.21.15.129:0/155175694 >> 172.21.15.129:0/155175694 conn(0x7f56d80fd8c0 msgr2=0x7f56d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 -- 172.21.15.129:0/155175694 shutdown_connections 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.097+0000 7f56dc996640 1 -- 172.21.15.129:0/155175694 wait complete. 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.098+0000 7f56dc996640 1 Processor -- start 2024-09-05T01:32:32.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.098+0000 7f56dc996640 1 -- start start 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 0x7f56d81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 0x7f56d819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8111e20 con 0x7f56d8101de0 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8111ca0 con 0x7f56d8106f10 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8111fa0 con 0x7f56d8102790 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:37836/0 (socket says 172.21.15.129:37836) 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 -- 172.21.15.129:0/3186181287 learned_addr learned my addr 172.21.15.129:0/3186181287 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d5d9b640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 0x7f56d81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d6d9d640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 0x7f56d819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:32.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 -- 172.21.15.129:0/3186181287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 msgr2=0x7f56d81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 0x7f56d81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 -- 172.21.15.129:0/3186181287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 msgr2=0x7f56d819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 0x7f56d819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 -- 172.21.15.129:0/3186181287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d819d310 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d6d9d640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 0x7f56d819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d5d9b640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 0x7f56d81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56d659c640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9f0 tx=0x7f56d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d000cd80 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d819d600 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56d00108e0 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d0018920 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.099+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d81a4ee0 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.100+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d81021c0 con 0x7f56d8101de0 2024-09-05T01:32:32.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.101+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f56d0010a90 con 0x7f56d8101de0 2024-09-05T01:32:32.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.102+0000 7f56c77fe640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 0x7f56c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:32.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.102+0000 7f56d5d9b640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 0x7f56c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:32.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.102+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f56d00cfd30 con 0x7f56d8101de0 2024-09-05T01:32:32.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.102+0000 7f56d5d9b640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 0x7f56c00a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009370 tx=0x7f56cc009440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:32.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.105+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56d009c880 con 0x7f56d8101de0 2024-09-05T01:32:32.309 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.308+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56d810a1b0 con 0x7f56d8101de0 2024-09-05T01:32:32.309 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:32 smithi129 ceph-mon[30979]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 200 KiB/s wr, 31 op/s 2024-09-05T01:32:32.310 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.309+0000 7f56c77fe640 1 -- 172.21.15.129:0/3186181287 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f56d009c180 con 0x7f56d8101de0 2024-09-05T01:32:32.310 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:32.310 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:32:32.311 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:32:32.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 msgr2=0x7f56c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:32.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 0x7f56c00a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009370 tx=0x7f56cc009440 comp rx=0 tx=0).stop 2024-09-05T01:32:32.312 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 msgr2=0x7f56d8198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f56d000e9f0 tx=0x7f56d000eeb0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 shutdown_connections 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f56c00a2b80 0x7f56c00a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f56d8106f10 0x7f56d819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f56d8102790 0x7f56d81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 --2- 172.21.15.129:0/3186181287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f56d8101de0 0x7f56d8198360 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 >> 172.21.15.129:0/3186181287 conn(0x7f56d80fd8c0 msgr2=0x7f56d80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 shutdown_connections 2024-09-05T01:32:32.313 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:32.311+0000 7f56dc996640 1 -- 172.21.15.129:0/3186181287 wait complete. 2024-09-05T01:32:32.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:32 smithi183 ceph-mon[34589]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 200 KiB/s wr, 31 op/s 2024-09-05T01:32:32.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:32 smithi194 ceph-mon[34941]: pgmap v869: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 200 KiB/s wr, 31 op/s 2024-09-05T01:32:33.107 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --set-id 599cc8ea --release --await' 2024-09-05T01:32:33.107 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --set-id 599cc8ea --release --await 2024-09-05T01:32:33.353 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:33.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:33 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3186181287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:33.474 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:33 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3186181287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:33.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:33 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3186181287' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 -- 172.21.15.129:0/2831179144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97981005a0 msgr2=0x7f979810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 --2- 172.21.15.129:0/2831179144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97981005a0 0x7f979810ea20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9790009ca0 tx=0x7f979002a8c0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 -- 172.21.15.129:0/2831179144 shutdown_connections 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 --2- 172.21.15.129:0/2831179144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f97981005a0 0x7f979810ea20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 --2- 172.21.15.129:0/2831179144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 0x7f9798100060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.852 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 --2- 172.21.15.129:0/2831179144 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9798106b80 0x7f9798106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 -- 172.21.15.129:0/2831179144 >> 172.21.15.129:0/2831179144 conn(0x7f97980fba90 msgr2=0x7f97980fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 -- 172.21.15.129:0/2831179144 shutdown_connections 2024-09-05T01:32:33.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.851+0000 7f979f334640 1 -- 172.21.15.129:0/2831179144 wait complete. 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 Processor -- start 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 -- start start 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 0x7f979819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97981005a0 0x7f979819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9798102ae0 con 0x7f9798106b80 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9798102960 con 0x7f97980ffc00 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979f334640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9798102c60 con 0x7f97981005a0 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979d8d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979d8d1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54126/0 (socket says 172.21.15.129:54126) 2024-09-05T01:32:33.863 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979d8d1640 1 -- 172.21.15.129:0/4256999431 learned_addr learned my addr 172.21.15.129:0/4256999431 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979d0d0640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 0x7f979819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.852+0000 7f979c8cf640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97981005a0 0x7f979819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 -- 172.21.15.129:0/4256999431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97981005a0 msgr2=0x7f979819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97981005a0 0x7f979819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 -- 172.21.15.129:0/4256999431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 msgr2=0x7f979819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 0x7f979819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 -- 172.21.15.129:0/4256999431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979819f490 con 0x7f9798106b80 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979d8d1640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9790003bd0 tx=0x7f9790003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f978e7fc640 1 -- 172.21.15.129:0/4256999431 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790004240 con 0x7f9798106b80 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f978e7fc640 1 -- 172.21.15.129:0/4256999431 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9790032e80 con 0x7f9798106b80 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979819f720 con 0x7f9798106b80 2024-09-05T01:32:33.864 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f978e7fc640 1 -- 172.21.15.129:0/4256999431 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979002f4c0 con 0x7f9798106b80 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.853+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97981a7060 con 0x7f9798106b80 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.855+0000 7f978e7fc640 1 -- 172.21.15.129:0/4256999431 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f97900305c0 con 0x7f9798106b80 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 Processor -- start 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 -- start start 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f978e7fc640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 0x7f97840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817980 0x556d50821d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d50824630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556d5077c620 con 0x556d50817060 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d50723e50 con 0x556d50822240 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f97a170d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556d507785e0 con 0x556d50817980 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f979d0d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f978e7fc640 1 -- 172.21.15.129:0/4256999431 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f97900f0c50 con 0x7f9798106b80 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f979c8cf640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 0x7f97840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f979d0d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54134/0 (socket says 172.21.15.129:54134) 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.856+0000 7f979d0d0640 1 -- 172.21.15.129:0/1144783628 learned_addr learned my addr 172.21.15.129:0/1144783628 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979c8cf640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817980 0x556d50821d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d8d1640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d50824630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 -- 172.21.15.129:0/1144783628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817980 msgr2=0x556d50821d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817980 0x556d50821d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 -- 172.21.15.129:0/1144783628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 msgr2=0x556d50824630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d50824630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 -- 172.21.15.129:0/1144783628 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d50714d40 con 0x556d50817060 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d8d1640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d50824630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979c8cf640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 0x7f97840a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9788005e10 tx=0x7f97880128d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f979d0d0640 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f979400b820 tx=0x7f979400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2b880317309f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.866 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f97837fe640 1 -- 172.21.15.129:0/1144783628 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979400d950 con 0x556d50817060 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f97837fe640 1 -- 172.21.15.129:0/1144783628 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9794019070 con 0x556d50817060 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.857+0000 7f97837fe640 1 -- 172.21.15.129:0/1144783628 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97940123e0 con 0x556d50817060 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- 172.21.15.129:0/1144783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 msgr2=0x556d50817440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f979400b820 tx=0x7f979400bce0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- 172.21.15.129:0/1144783628 shutdown_connections 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d50824630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817980 0x556d50821d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 --2- 172.21.15.129:0/1144783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817060 0x556d50817440 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- 172.21.15.129:0/1144783628 >> 172.21.15.129:0/1144783628 conn(0x556d508135f0 msgr2=0x556d508139d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- 172.21.15.129:0/1144783628 shutdown_connections 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- 172.21.15.129:0/1144783628 wait complete. 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 -- start start 2024-09-05T01:32:33.867 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.858+0000 7f97a170d740 1 Processor -- start 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5081f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556d50824e10 con 0x556d50817980 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d50704f10 con 0x556d50822240 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x556d50824f90 con 0x556d50817060 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 10 client.0 _ll_register_callbacks cb 0x556d50728db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54136/0 (socket says 172.21.15.129:54136) 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 -- 172.21.15.129:0/2840978578 learned_addr learned my addr 172.21.15.129:0/2840978578 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979d0d0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5081f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979d8d1640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.868 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 -- 172.21.15.129:0/2840978578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 msgr2=0x556d5081f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5081f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 -- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 msgr2=0x556d5081fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 -- 172.21.15.129:0/2840978578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d5048b580 con 0x556d50817980 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f979c8cf640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9788004460 tx=0x7f9788018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3abbfd492d29a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f9780ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788024070 con 0x556d50817980 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9788010980 con 0x556d50817980 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.859+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9794019070 con 0x556d50817980 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9788030050 con 0x556d50817980 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 10 client.17181 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:33.869 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d50555960 con 0x556d50817980 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9794019070 con 0x556d50817980 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f978800f040 con 0x556d50817980 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 10 client.17181 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 10 client.17181 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 20 client.17181 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d50831c30 0x556d50834090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 4 client.17181 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d50789b90 con 0x556d50831c30 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.860+0000 7f979d0d0640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d50831c30 0x556d50834090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.861+0000 7f979d0d0640 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d50831c30 0x556d50834090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116f03fdd92274f5 server_cookie=102e6079e8c6a384 in_seq=0 out_seq=0 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.861+0000 7f9780ff9640 10 client.17181 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.861+0000 7f9780ff9640 1 -- 172.21.15.129:0/2840978578 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d50789b90 con 0x556d50831c30 2024-09-05T01:32:33.870 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.861+0000 7f9780ff9640 10 client.17181 handle_command_reply: tid=0 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.861+0000 7f97a170d740 1 client.17181 shutdown 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 msgr2=0x556d5081f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9788004460 tx=0x7f9788018000 comp rx=0 tx=0).stop 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 shutdown_connections 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d50831c30 0x556d50834090 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5081f5d0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 --2- 172.21.15.129:0/2840978578 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5081f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 >> 172.21.15.129:0/2840978578 conn(0x556d508135f0 msgr2=0x556d508192d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f9780ff9640 0 client.17181 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 shutdown_connections 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 1 -- 172.21.15.129:0/2840978578 wait complete. 2024-09-05T01:32:33.871 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.862+0000 7f97a170d740 20 client.17181 trim_cache size 0 max 16384 2024-09-05T01:32:33.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 Processor -- start 2024-09-05T01:32:33.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 -- start start 2024-09-05T01:32:33.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d50839960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.880 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97981b3840 con 0x556d50817980 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97981b1110 con 0x556d50822240 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f97a170d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97981aef60 con 0x556d50817060 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54148/0 (socket says 172.21.15.129:54148) 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 -- 172.21.15.129:0/1051494069 learned_addr learned my addr 172.21.15.129:0/1051494069 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979d8d1640 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979d0d0640 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d50839960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 -- 172.21.15.129:0/1051494069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 msgr2=0x556d50839960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d50839960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 -- 172.21.15.129:0/1051494069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 msgr2=0x556d5081c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.881 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 -- 172.21.15.129:0/1051494069 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d508177b0 con 0x556d50817980 2024-09-05T01:32:33.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f979c8cf640 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f978801ec50 tx=0x7f9788004090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ff00b5a31681fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.879+0000 7f9782ffd640 1 -- 172.21.15.129:0/1051494069 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788039070 con 0x556d50817980 2024-09-05T01:32:33.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f9782ffd640 1 -- 172.21.15.129:0/1051494069 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978801fe50 con 0x556d50817980 2024-09-05T01:32:33.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f9782ffd640 1 -- 172.21.15.129:0/1051494069 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788024040 con 0x556d50817980 2024-09-05T01:32:33.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 -- 172.21.15.129:0/1051494069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 msgr2=0x556d50839ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f978801ec50 tx=0x7f9788004090 comp rx=0 tx=0).stop 2024-09-05T01:32:33.885 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 -- 172.21.15.129:0/1051494069 shutdown_connections 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d5081c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d50839ea0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 --2- 172.21.15.129:0/1051494069 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d50839960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 -- 172.21.15.129:0/1051494069 >> 172.21.15.129:0/1051494069 conn(0x556d508135f0 msgr2=0x556d508208f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 -- 172.21.15.129:0/1051494069 shutdown_connections 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.880+0000 7f97a170d740 1 -- 172.21.15.129:0/1051494069 wait complete. 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 -- start start 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 Processor -- start 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5082a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d508271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.886 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f97981af5d0 con 0x556d50817980 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97981b0a40 con 0x556d50822240 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f97981afca0 con 0x556d50817060 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f97a170d740 10 client.0 _ll_register_callbacks cb 0x7f9788004100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54160/0 (socket says 172.21.15.129:54160) 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 -- 172.21.15.129:0/604947991 learned_addr learned my addr 172.21.15.129:0/604947991 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979d8d1640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d508271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979d0d0640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5082a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 -- 172.21.15.129:0/604947991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 msgr2=0x556d5082a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5082a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 -- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 msgr2=0x556d508271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d508271b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979c8cf640 1 -- 172.21.15.129:0/604947991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d508276f0 con 0x556d50817980 2024-09-05T01:32:33.887 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.881+0000 7f979d0d0640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5082a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f979c8cf640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9788037040 tx=0x7f9788010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7749ae47af7cfb6b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788039030 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9788010780 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f978801fe50 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9788024040 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 10 client.17187 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556d5089d0e0 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978801fe50 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7f9788029440 con 0x556d50817980 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 10 client.17187 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 10 client.17187 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:33.888 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 20 client.17187 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d5089d790 0x556d5089fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 4 client.17187 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "599cc8ea", "release": true, "await": true}] 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.882+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "set_id": "599cc8ea", "release": true, "await": true}) -- 0x556d5080e950 con 0x556d5089d790 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.883+0000 7f979d0d0640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d5089d790 0x556d5089fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.883+0000 7f9780ff9640 10 client.17187 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:33.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.883+0000 7f979d0d0640 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d5089d790 0x556d5089fbb0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75e327f8511f7880 server_cookie=53ed9657569d987b in_seq=0 out_seq=0 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.970+0000 7f9780ff9640 1 -- 172.21.15.129:0/604947991 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556d5080e950 con 0x556d5089d790 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.970+0000 7f9780ff9640 10 client.17187 handle_command_reply: tid=0 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.970+0000 7f97a170d740 1 client.17187 shutdown 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.970+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 msgr2=0x556d5082a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.970+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f9788037040 tx=0x7f9788010f90 comp rx=0 tx=0).stop 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 shutdown_connections 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x556d5089d790 0x556d5089fbb0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d50822240 0x556d508271b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.972 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d50817980 0x556d5082a720 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 --2- 172.21.15.129:0/604947991 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x556d50817060 0x556d5082a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f9780ff9640 0 client.17187 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 >> 172.21.15.129:0/604947991 conn(0x556d508135f0 msgr2=0x556d50811f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 shutdown_connections 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 1 -- 172.21.15.129:0/604947991 wait complete. 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.971+0000 7f97a170d740 20 client.17187 trim_cache size 0 max 16384 2024-09-05T01:32:33.973 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 68, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": { 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "599cc8ea": { 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "version": 68, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age_ref": 0.0, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "timeout": 90.0, 2024-09-05T01:32:33.974 INFO:teuthology.orchestra.run.smithi129.stdout: "expiration": 150.0, 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "members": { 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "file:/": { 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "excluded": false, 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "state": { 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RELEASED", 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: "age": 0.0 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:33.975 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:33.976 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 msgr2=0x7f97840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 0x7f97840a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9788005e10 tx=0x7f97880128d0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 msgr2=0x7f979819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f9790003bd0 tx=0x7f9790003c00 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 shutdown_connections 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f97840a2b80 0x7f97840a5040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9798106b80 0x7f979819edb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f97981005a0 0x7f979819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 --2- 172.21.15.129:0/4256999431 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97980ffc00 0x7f979819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 >> 172.21.15.129:0/4256999431 conn(0x7f97980fba90 msgr2=0x7f97980fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 shutdown_connections 2024-09-05T01:32:33.977 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:33.972+0000 7f979f334640 1 -- 172.21.15.129:0/4256999431 wait complete. 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:transaction type: 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:scaling factor: 32 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:query mode: simple 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:number of clients: 100 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:number of threads: 4 2024-09-05T01:32:34.206 INFO:teuthology.orchestra.run.smithi129.stdout:duration: 900 s 2024-09-05T01:32:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:number of transactions actually processed: 590583 2024-09-05T01:32:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:latency average = 157.224 ms 2024-09-05T01:32:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:latency stddev = 1093.596 ms 2024-09-05T01:32:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:tps = 635.719615 (including connections establishing) 2024-09-05T01:32:34.207 INFO:teuthology.orchestra.run.smithi129.stdout:tps = 635.727132 (excluding connections establishing) 2024-09-05T01:32:34.209 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-05T01:32:34.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:34 smithi183 ceph-mon[34589]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 185 KiB/s wr, 30 op/s 2024-09-05T01:32:34.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:34 smithi129 ceph-mon[30979]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 185 KiB/s wr, 30 op/s 2024-09-05T01:32:34.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:34 smithi194 ceph-mon[34941]: pgmap v870: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 185 KiB/s wr, 30 op/s 2024-09-05T01:32:34.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '599cc8ea', total seconds elapsed: 33.5 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.225+0000 7fee848f1640 1 -- 172.21.15.129:0/3478749042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee801025f0 msgr2=0x7fee80108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.225+0000 7fee848f1640 1 --2- 172.21.15.129:0/3478749042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee801025f0 0x7fee80108e80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fee78009ca0 tx=0x7fee7802a8c0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.225+0000 7fee848f1640 1 -- 172.21.15.129:0/3478749042 shutdown_connections 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.225+0000 7fee848f1640 1 --2- 172.21.15.129:0/3478749042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee801025f0 0x7fee80108e80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.225+0000 7fee848f1640 1 --2- 172.21.15.129:0/3478749042 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee80101c30 0x7fee801020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 --2- 172.21.15.129:0/3478749042 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.229 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- 172.21.15.129:0/3478749042 >> 172.21.15.129:0/3478749042 conn(0x7fee800fc1e0 msgr2=0x7fee800fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- 172.21.15.129:0/3478749042 shutdown_connections 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- 172.21.15.129:0/3478749042 wait complete. 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 Processor -- start 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- start start 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 0x7fee8019f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c5a0 con 0x7fee80101c30 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c420 con 0x7fee80100a40 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.226+0000 7fee848f1640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fee8010c720 con 0x7fee801025f0 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.230 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54170/0 (socket says 172.21.15.129:54170) 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7e59c640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 -- 172.21.15.129:0/391789596 learned_addr learned my addr 172.21.15.129:0/391789596 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 0x7fee8019f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 -- 172.21.15.129:0/391789596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 msgr2=0x7fee8019f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 0x7fee8019f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 -- 172.21.15.129:0/391789596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 msgr2=0x7fee80197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 -- 172.21.15.129:0/391789596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8019f960 con 0x7fee80101c30 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 0x7fee8019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7e59c640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e9f0 tx=0x7fee7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee6f7fe640 1 -- 172.21.15.129:0/391789596 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7400cd80 con 0x7fee80101c30 2024-09-05T01:32:35.231 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee6f7fe640 1 -- 172.21.15.129:0/391789596 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee740108e0 con 0x7fee80101c30 2024-09-05T01:32:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee6f7fe640 1 -- 172.21.15.129:0/391789596 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74018920 con 0x7fee80101c30 2024-09-05T01:32:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8019fc50 con 0x7fee80101c30 2024-09-05T01:32:35.232 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.227+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a01a0 con 0x7fee80101c30 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee6f7fe640 1 -- 172.21.15.129:0/391789596 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fee74010a90 con 0x7fee80101c30 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 Processor -- start 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 -- start start 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd86da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd87370 0x55a19bd89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a19bce52f0 con 0x55a19bd87370 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a19bc8d270 con 0x55a19bd822e0 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee8768b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a19bce12b0 con 0x55a19bd819a0 2024-09-05T01:32:35.238 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.230+0000 7fee6f7fe640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 0x7fee680a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd87370 0x55a19bd89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 0x7fee680a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd86da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd86da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.129:45532/0 (socket says 172.21.15.129:45532) 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7e59c640 1 -- 172.21.15.129:0/679772935 learned_addr learned my addr 172.21.15.129:0/679772935 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.194:3300/0 says I am v2:172.21.15.129:52130/0 (socket says 172.21.15.129:52130) 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee6f7fe640 1 -- 172.21.15.129:0/391789596 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee740cf7d0 con 0x7fee80101c30 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 -- 172.21.15.129:0/679772935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 msgr2=0x55a19bd86da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd86da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 -- 172.21.15.129:0/679772935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd87370 msgr2=0x55a19bd89830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.239 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd87370 0x55a19bd89830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 -- 172.21.15.129:0/679772935 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19bc8c340 con 0x55a19bd819a0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee78006010 tx=0x7fee78032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89e7d67b7fc78b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 0x7fee680a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fee78009a60 tx=0x7fee78043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee6cff9640 1 -- 172.21.15.129:0/679772935 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7804e070 con 0x55a19bd819a0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee6cff9640 1 -- 172.21.15.129:0/679772935 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee78036040 con 0x55a19bd819a0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.231+0000 7fee6cff9640 1 -- 172.21.15.129:0/679772935 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780047e0 con 0x55a19bd819a0 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 -- 172.21.15.129:0/679772935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 msgr2=0x55a19bd81da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fee78006010 tx=0x7fee78032ce0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.240 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 -- 172.21.15.129:0/679772935 shutdown_connections 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd87370 0x55a19bd89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd86da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 --2- 172.21.15.129:0/679772935 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd819a0 0x55a19bd81da0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 -- 172.21.15.129:0/679772935 >> 172.21.15.129:0/679772935 conn(0x55a19bd7d010 msgr2=0x55a19bd7d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 -- 172.21.15.129:0/679772935 shutdown_connections 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.232+0000 7fee8768b740 1 -- 172.21.15.129:0/679772935 wait complete. 2024-09-05T01:32:35.241 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 -- start start 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 Processor -- start 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd8bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd931b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a19bd8a2e0 con 0x55a19bd822e0 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a19bc6e290 con 0x55a19bd819a0 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x55a19bd8a460 con 0x55a19bd87370 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee8768b740 10 client.0 _ll_register_callbacks cb 0x55a19bc9f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54194/0 (socket says 172.21.15.129:54194) 2024-09-05T01:32:35.242 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 -- 172.21.15.129:0/603230584 learned_addr learned my addr 172.21.15.129:0/603230584 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7e59c640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd8bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd931b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 -- 172.21.15.129:0/603230584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 msgr2=0x55a19bd931b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd931b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 -- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 msgr2=0x55a19bd8bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd8bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 -- 172.21.15.129:0/603230584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19b9f8210 con 0x55a19bd822e0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7e59c640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fee74098080 tx=0x7fee7401d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f416c17033bb432 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.233+0000 7fee65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7409bc20 con 0x55a19bd822e0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee78036040 con 0x55a19bd822e0 2024-09-05T01:32:35.243 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee740a4400 con 0x55a19bd822e0 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee7409ffd0 con 0x55a19bd822e0 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 10 client.17199 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a19bac4620 con 0x55a19bd822e0 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee78036040 con 0x55a19bd822e0 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fee74097970 con 0x55a19bd822e0 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 10 client.17199 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 20 client.17199 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19bd97920 0x55a19bd99d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 4 client.17199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a19bcf2a20 con 0x55a19bd97920 2024-09-05T01:32:35.244 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.234+0000 7fee7e59c640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19bd97920 0x55a19bd99d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.235+0000 7fee65ffb640 10 client.17199 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.235+0000 7fee7e59c640 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19bd97920 0x55a19bd99d10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b64e2a70b29f4c server_cookie=cecdb39d0a62a700 in_seq=0 out_seq=0 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.235+0000 7fee65ffb640 1 -- 172.21.15.129:0/603230584 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a19bcf2a20 con 0x55a19bd97920 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.235+0000 7fee65ffb640 10 client.17199 handle_command_reply: tid=0 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.235+0000 7fee8768b740 1 client.17199 shutdown 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 msgr2=0x55a19bd8c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fee74098080 tx=0x7fee7401d400 comp rx=0 tx=0).stop 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 shutdown_connections 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19bd97920 0x55a19bd99d10 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd931b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd8c170 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 --2- 172.21.15.129:0/603230584 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd8bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.245 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee65ffb640 0 client.17199 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 >> 172.21.15.129:0/603230584 conn(0x55a19bd7d010 msgr2=0x55a19bd7e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 shutdown_connections 2024-09-05T01:32:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 1 -- 172.21.15.129:0/603230584 wait complete. 2024-09-05T01:32:35.246 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.236+0000 7fee8768b740 20 client.17199 trim_cache size 0 max 16384 2024-09-05T01:32:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.250+0000 7fee8768b740 1 Processor -- start 2024-09-05T01:32:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.250+0000 7fee8768b740 1 -- start start 2024-09-05T01:32:35.252 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd7c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ad200 con 0x55a19bd822e0 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aaad0 con 0x55a19bd819a0 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee8768b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a8920 con 0x55a19bd87370 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7e59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd7c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54206/0 (socket says 172.21.15.129:54206) 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 -- 172.21.15.129:0/2582872108 learned_addr learned my addr 172.21.15.129:0/2582872108 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 -- 172.21.15.129:0/2582872108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 msgr2=0x55a19bd9e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd9e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.253 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 -- 172.21.15.129:0/2582872108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 msgr2=0x55a19bd7c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd7c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 -- 172.21.15.129:0/2582872108 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a19bd9a340 con 0x55a19bd822e0 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7e59c640 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd7c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fee740985f0 tx=0x7fee7409a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ac4a015bad3039 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee67fff640 1 -- 172.21.15.129:0/2582872108 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee74092030 con 0x55a19bd822e0 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee67fff640 1 -- 172.21.15.129:0/2582872108 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee74010de0 con 0x55a19bd822e0 2024-09-05T01:32:35.254 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.251+0000 7fee67fff640 1 -- 172.21.15.129:0/2582872108 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee740a4400 con 0x55a19bd822e0 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 -- 172.21.15.129:0/2582872108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 msgr2=0x55a19bd93170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fee740985f0 tx=0x7fee7409a130 comp rx=0 tx=0).stop 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 -- 172.21.15.129:0/2582872108 shutdown_connections 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd9e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd822e0 0x55a19bd93170 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.257 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 --2- 172.21.15.129:0/2582872108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd819a0 0x55a19bd7c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 -- 172.21.15.129:0/2582872108 >> 172.21.15.129:0/2582872108 conn(0x55a19bd7d010 msgr2=0x55a19bd7b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 -- 172.21.15.129:0/2582872108 shutdown_connections 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.252+0000 7fee8768b740 1 -- 172.21.15.129:0/2582872108 wait complete. 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 -- start start 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 Processor -- start 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd91720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd92730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a8f90 con 0x55a19bd819a0 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aa400 con 0x55a19bd822e0 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a9660 con 0x55a19bd87370 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee8768b740 10 client.0 _ll_register_callbacks cb 0x7fee74097f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:35.258 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:54212/0 (socket says 172.21.15.129:54212) 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 -- 172.21.15.129:0/234858815 learned_addr learned my addr 172.21.15.129:0/234858815 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd91720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7ed9d640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd92730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 -- 172.21.15.129:0/234858815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 msgr2=0x55a19bd92730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd92730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 -- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 msgr2=0x55a19bd91720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd91720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7e59c640 1 -- 172.21.15.129:0/234858815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a19bd92d40 con 0x55a19bd819a0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.253+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd91720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee7e59c640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55a19bd7de90 tx=0x7fee70010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698fe585260d1bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:35.259 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7000fe00 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fee700047a0 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a19bcecd70 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee7002f050 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 10 client.17205 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a19be02f00 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a19bcecd70 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fee700273f0 con 0x55a19bd819a0 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 10 client.17205 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 10 client.17205 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 20 client.17205 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19be038e0 0x55a19be05d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:35.260 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 4 client.17205 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:32:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.254+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a19bc69900 con 0x55a19be038e0 2024-09-05T01:32:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.255+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19be038e0 0x55a19be05d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.255+0000 7fee65ffb640 10 client.17205 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:35.261 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.255+0000 7fee7dd9b640 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19be038e0 0x55a19be05d00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bdb8723e39de13a server_cookie=c61f246034a0ff76 in_seq=0 out_seq=0 2024-09-05T01:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee65ffb640 1 -- 172.21.15.129:0/234858815 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a19bc69900 con 0x55a19be038e0 2024-09-05T01:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee65ffb640 10 client.17205 handle_command_reply: tid=0 2024-09-05T01:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 client.17205 shutdown 2024-09-05T01:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 msgr2=0x55a19bd911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x55a19bd7de90 tx=0x7fee70010ec0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 shutdown_connections 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x55a19be038e0 0x55a19be05d00 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x55a19bd87370 0x55a19bd92730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a19bd822e0 0x55a19bd91720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee65ffb640 0 client.17205 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 --2- 172.21.15.129:0/234858815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a19bd819a0 0x55a19bd911e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 >> 172.21.15.129:0/234858815 conn(0x55a19bd7d010 msgr2=0x55a19bd7c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 shutdown_connections 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.309+0000 7fee8768b740 1 -- 172.21.15.129:0/234858815 wait complete. 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee8768b740 20 client.17205 trim_cache size 0 max 16384 2024-09-05T01:32:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:32:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:32:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:32:35.312 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 msgr2=0x7fee680a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 0x7fee680a50f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fee78009a60 tx=0x7fee78043000 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 msgr2=0x7fee80198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e9f0 tx=0x7fee7400eeb0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 shutdown_connections 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fee680a2c30 0x7fee680a50f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fee801025f0 0x7fee8019f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fee80101c30 0x7fee80198280 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 --2- 172.21.15.129:0/391789596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee80100a40 0x7fee80197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 >> 172.21.15.129:0/391789596 conn(0x7fee800fc1e0 msgr2=0x7fee800fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 shutdown_connections 2024-09-05T01:32:35.313 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:35.310+0000 7fee848f1640 1 -- 172.21.15.129:0/391789596 wait complete. 2024-09-05T01:32:36.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:36 smithi183 ceph-mon[34589]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-09-05T01:32:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:36 smithi129 ceph-mon[30979]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-09-05T01:32:36.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:36 smithi194 ceph-mon[34941]: pgmap v871: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 167 KiB/s wr, 28 op/s 2024-09-05T01:32:38.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:38 smithi183 ceph-mon[34589]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 372 KiB/s wr, 45 op/s 2024-09-05T01:32:38.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:38 smithi129 ceph-mon[30979]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 372 KiB/s wr, 45 op/s 2024-09-05T01:32:38.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:38 smithi194 ceph-mon[34941]: pgmap v872: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 372 KiB/s wr, 45 op/s 2024-09-05T01:32:40.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:40 smithi183 ceph-mon[34589]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-09-05T01:32:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:40 smithi129 ceph-mon[30979]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-09-05T01:32:40.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:40 smithi194 ceph-mon[34941]: pgmap v873: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.9 MiB/s wr, 228 op/s 2024-09-05T01:32:42.392 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:42 smithi129 ceph-mon[30979]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-09-05T01:32:42.398 DEBUG:teuthology.orchestra.run.smithi129:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-05T01:32:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:42 smithi183 ceph-mon[34589]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-09-05T01:32:42.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:42 smithi194 ceph-mon[34941]: pgmap v874: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 3.9 MiB/s wr, 443 op/s 2024-09-05T01:32:42.733 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/: 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:total 2 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841852 Sep 5 01:15 . 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1450841852 Sep 5 01:15 .. 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 data 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data: 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:total 46 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 . 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841852 Sep 5 01:15 .. 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 base 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 5 01:15 current_logfiles 2024-09-05T01:32:42.734 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Sep 5 01:15 global 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1676 Sep 5 01:15 log 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_commit_ts 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_dynshmem 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 5 01:15 pg_hba.conf 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 5 01:15 pg_ident.conf 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:32 pg_logical 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 pg_multixact 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_notify 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_replslot 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_serial 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_snapshots 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 5 01:32 pg_stat 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:32 pg_stat_tmp 2024-09-05T01:32:42.735 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 pg_subtrans 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_tblspc 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 pg_twophase 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 Sep 5 01:32 pg_wal 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Sep 5 01:15 pg_xact 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 5 01:15 postgresql.auto.conf 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 5 01:15 postgresql.conf 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 5 01:15 postmaster.opts 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base: 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:total 3 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 . 2024-09-05T01:32:42.736 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 1 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 13493 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 561697327 Sep 5 01:22 13494 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:17 pgsql_tmp 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:total 7778 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 . 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 .. 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 1247 2024-09-05T01:32:42.737 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1247_vm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:15 1249 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1249_fsm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1249_vm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:15 1259 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1259_vm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:32:42.738 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:32:42.739 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2579 2024-09-05T01:32:42.740 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:32:42.741 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606_vm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:15 2608 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2608_vm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:32:42.742 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2610 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2610_vm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:32:42.743 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 5 01:15 2619 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2619_fsm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2619_vm 2024-09-05T01:32:42.744 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:15 2658 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 2659 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2662 2024-09-05T01:32:42.745 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2663 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2664 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2665 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2666 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2667 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:15 2673 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:15 2674 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2678 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2679 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:32:42.746 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:32:42.747 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2696 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2703 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2704 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:32:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:32:42.749 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:32:42.750 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:32:42.751 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3455 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:32:42.752 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:32:42.753 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:32:42.754 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:32:42.755 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:32:42.756 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:32:42.756 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:32:42.756 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:32:42.956 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:32:42.957 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:32:42.958 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:total 7778 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7963139 Sep 5 01:15 . 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 .. 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 1247 2024-09-05T01:32:42.959 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1247_vm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:15 1249 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1249_fsm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1249_vm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:15 1259 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1259_vm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:32:42.960 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:32:42.961 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:32:42.962 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2579 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:32:42.963 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2606_vm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:15 2608 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2608_vm 2024-09-05T01:32:42.964 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2610 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2610_vm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:32:42.965 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 5 01:15 2619 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2619_fsm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2619_vm 2024-09-05T01:32:42.966 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:15 2658 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:15 2659 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2662 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2663 2024-09-05T01:32:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2664 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2665 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2666 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2667 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:15 2673 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:15 2674 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2678 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2679 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:32:42.968 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2696 2024-09-05T01:32:42.969 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2703 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2704 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:32:42.970 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:32:42.971 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:32:42.972 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:32:42.973 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3455 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:32:42.974 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:32:42.975 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:32:42.976 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:32:42.977 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:32:42.978 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:32:43.193 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:32:43.194 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:32:43.195 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:total 548535 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 561697327 Sep 5 01:22 . 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 .. 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 112 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 113 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:17 1247 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1247_fsm 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 1247_vm 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 5 01:17 1249 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:17 1249_fsm 2024-09-05T01:32:43.196 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 1249_vm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 5 01:15 1255 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1255_fsm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1255_vm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 5 01:32 1259 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1259_fsm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 1259_vm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 5 01:15 13344 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13344_fsm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13344_vm 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13346 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13348 2024-09-05T01:32:43.197 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13349_fsm 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13349_vm 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13351 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13353 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13354_fsm 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13354_vm 2024-09-05T01:32:43.200 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13356 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13358 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 13359_fsm 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13359_vm 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 13361 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 13363 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1417 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 1418 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 443219968 Sep 5 01:32 16396 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 5 01:29 16396_fsm 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:31 16396_vm 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3014656 Sep 5 01:32 16397 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:32 16397_fsm 2024-09-05T01:32:43.201 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 16397_vm 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3612672 Sep 5 01:32 16399 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:32 16399_fsm 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 16399_vm 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:32 16400 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:32 16402 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 5 01:32 16404 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 31481856 Sep 5 01:32 16406 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:32 16406_fsm 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:31 16406_vm 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 174 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 175 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2187 2024-09-05T01:32:43.202 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2224 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2328 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2336 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2337 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2579 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2600 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2600_fsm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2600_vm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2601_fsm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2601_vm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2602 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2602_fsm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2602_vm 2024-09-05T01:32:43.203 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2603 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2603_fsm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2603_vm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2604 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2605 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2605_fsm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2605_vm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 2606 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2606_fsm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 2606_vm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2607 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2607_fsm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2607_vm 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 5 01:17 2608 2024-09-05T01:32:43.204 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2608_fsm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 2608_vm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 5 01:15 2609 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2609_fsm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2609_vm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:17 2610 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2610_fsm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 2610_vm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2611 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2612_fsm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2612_vm 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2613 2024-09-05T01:32:43.205 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2615_fsm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2615_vm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2616 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2616_fsm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2616_vm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 5 01:15 2617 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2617_fsm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2617_vm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 5 01:15 2618 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2618_fsm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2618_vm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 5 01:32 2619 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:19 2619_fsm 2024-09-05T01:32:43.206 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:17 2619_vm 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2620 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2650 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2651 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2652 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2653 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2654 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2655 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2656 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2657 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 5 01:17 2658 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 5 01:17 2659 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2660 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2661 2024-09-05T01:32:43.207 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:17 2662 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:17 2663 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2664 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2665 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2666 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2667 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2668 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2669 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2670 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 5 01:17 2673 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 5 01:17 2674 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 5 01:15 2675 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2678 2024-09-05T01:32:43.208 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:17 2679 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2680 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2681 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2682 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2683 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2684 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2685 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2686 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2687 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 2688 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 5 01:15 2689 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 5 01:15 2690 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 5 01:15 2691 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2692 2024-09-05T01:32:43.209 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2693 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:20 2696 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2699 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2701 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2702 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:17 2703 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:17 2704 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2753 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2753_fsm 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2753_vm 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2754 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2755 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2756 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 2757 2024-09-05T01:32:43.210 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2830 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2831 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2832 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2833 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2834 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2835 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2836 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2837 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 5 01:15 2838 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2838_fsm 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2838_vm 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2839 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2840_fsm 2024-09-05T01:32:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2840_vm 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2841 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2995 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2996 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3079_fsm 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3079_vm 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3080 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3081 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:15 3085 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3118 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3119 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 5 01:15 3164 2024-09-05T01:32:43.212 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3256 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3257 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3258 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3350 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3351 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3379 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3380 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3381 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3394 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3394_fsm 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3394_vm 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3395 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3429 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3430 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3431 2024-09-05T01:32:43.213 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3433 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3439 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3440 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 5 01:17 3455 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 5 01:15 3456 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3456_fsm 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3456_vm 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3466 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3467 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3468 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3501 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3502 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3503 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3534 2024-09-05T01:32:43.214 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3541_fsm 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3541_vm 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3542 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3574 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3575 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3576 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3596 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3597 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3598 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3599 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3600_fsm 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3600_vm 2024-09-05T01:32:43.215 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3601_fsm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3601_vm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3602_fsm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3602_vm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3603_fsm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3603_vm 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3604 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3605 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3606 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3607 2024-09-05T01:32:43.216 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3608 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 5 01:15 3609 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3712 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 3764_fsm 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3764_vm 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3766 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 3767 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3997 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4143 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4144 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4145 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4146 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4147 2024-09-05T01:32:43.217 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4148 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4149 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4150 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4151 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4152 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4153 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4154 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4155 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4156 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4157 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4158 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4159 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4160 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4161 2024-09-05T01:32:43.218 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4162 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4163 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4164 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4165 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4166 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4167 2024-09-05T01:32:43.219 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4168 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4169 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4170 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4171 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4172 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4173 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4174 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 5002 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 548 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 549 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6102 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6104 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6106 2024-09-05T01:32:43.220 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6110 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6111 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6112 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6113 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6117 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 826 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 827 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 828 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 5 01:15 pg_internal.init 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 5 01:15 PG_VERSION 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.234 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:17 . 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 577623605 Sep 5 01:16 .. 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/global: 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:total 561 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 572520 Sep 5 01:15 . 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1213 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1213_fsm 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1213_vm 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1214 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1214_fsm 2024-09-05T01:32:43.235 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1214_vm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 1232 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 1233 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1260 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1260_fsm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1260_vm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1261 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1261_fsm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1261_vm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1262 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 1262_fsm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 1262_vm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2396 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 5 01:15 2396_fsm 2024-09-05T01:32:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2396_vm 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2397 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2671 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2672 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2676 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2677 2024-09-05T01:32:43.237 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2694 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2695 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2697 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 2698 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2846 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2847 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2964 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2965 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 2966 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 2967 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 3592 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 3593 2024-09-05T01:32:43.247 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4060 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4061 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4175 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4176 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4177 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4178 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4181 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4182 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4183 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4184 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 4185 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 4186 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6000 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6001 2024-09-05T01:32:43.248 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6002 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 6100 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6114 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 6115 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 pg_control 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 5 01:15 pg_filenode.map 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 5 01:15 pg_internal.init 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/log: 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:total 3 2024-09-05T01:32:43.249 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1676 Sep 5 01:15 . 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1747 Sep 5 01:32 postgresql-Thu.log 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.250 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.251 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-05T01:32:43.251 INFO:teuthology.orchestra.run.smithi129.stdout:total 3 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:32 . 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 mappings 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:32 replorigin_checkpoint 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 snapshots 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:32 .. 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.285 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 5 01:32 .. 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:total 2 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 . 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 members 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 offsets 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-05T01:32:43.286 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 .. 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 0000 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:15 . 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 5 01:15 .. 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:21 0000 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.287 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-05T01:32:43.288 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:total 16 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 14082 Sep 5 01:32 . 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 5 01:32 db_0.stat 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 5 01:32 db_13494.stat 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 5 01:32 global.stat 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.289 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:32 . 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:total 9 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 . 2024-09-05T01:32:43.290 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.327 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 5 01:32 0009 2024-09-05T01:32:43.328 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.328 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-05T01:32:43.328 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.328 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.328 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.329 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:total 851970 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 Sep 5 01:32 . 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:32 00000001000000000000006B 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:21 00000001000000000000006C 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:22 00000001000000000000006D 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:24 00000001000000000000006E 2024-09-05T01:32:43.330 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:21 00000001000000000000006F 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:23 000000010000000000000070 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:22 000000010000000000000071 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000072 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 000000010000000000000073 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000074 2024-09-05T01:32:43.331 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 000000010000000000000075 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:28 000000010000000000000076 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:28 000000010000000000000077 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000078 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000079 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 00000001000000000000007A 2024-09-05T01:32:43.332 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000007B 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000007C 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:30 00000001000000000000007D 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 00000001000000000000007E 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000007F 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000080 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:29 000000010000000000000081 2024-09-05T01:32:43.333 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000082 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000083 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:28 000000010000000000000084 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000085 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:28 000000010000000000000086 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:29 000000010000000000000087 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000088 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000089 2024-09-05T01:32:43.334 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000008A 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 00000001000000000000008B 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000008C 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:27 00000001000000000000008D 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000008E 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 00000001000000000000008F 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:25 000000010000000000000090 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:26 000000010000000000000091 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:30 000000010000000000000092 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000093 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000094 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000095 2024-09-05T01:32:43.335 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:32 000000010000000000000096 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000097 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000098 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 000000010000000000000099 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:30 00000001000000000000009A 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 00000001000000000000009B 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 00000001000000000000009C 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 00000001000000000000009D 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 5 01:31 00000001000000000000009E 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 archive_status 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:total 1 2024-09-05T01:32:43.336 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 5 01:15 . 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 872415232 Sep 5 01:32 .. 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:total 153 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Sep 5 01:15 . 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 1450841745 Sep 5 01:32 .. 2024-09-05T01:32:43.337 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Sep 5 01:32 0000 2024-09-05T01:32:43.338 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-05T01:32:43.350 INFO:tasks.quiescer:joining Quiescers 2024-09-05T01:32:43.350 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-05T01:32:43.350 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:32:43.596 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:44.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:44 smithi183 ceph-mon[34589]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-09-05T01:32:44.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:44 smithi129 ceph-mon[30979]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-09-05T01:32:44.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:44 smithi194 ceph-mon[34941]: pgmap v875: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 5.0 MiB/s wr, 548 op/s 2024-09-05T01:32:46.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:46 smithi183 ceph-mon[34589]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 738 op/s 2024-09-05T01:32:46.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:46 smithi129 ceph-mon[30979]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 738 op/s 2024-09-05T01:32:46.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:46 smithi194 ceph-mon[34941]: pgmap v876: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 738 op/s 2024-09-05T01:32:47.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:47 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1083022528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:47.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:47 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1083022528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:47.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:47 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1083022528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:32:47.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 -- 172.21.15.129:0/4207178771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8102790 msgr2=0x7f2fd81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:47.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/4207178771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8102790 0x7f2fd81067e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009900 tx=0x7f2fc802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:47.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 -- 172.21.15.129:0/4207178771 shutdown_connections 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/4207178771 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8106f10 0x7f2fd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/4207178771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8102790 0x7f2fd81067e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/4207178771 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8101de0 0x7f2fd81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 -- 172.21.15.129:0/4207178771 >> 172.21.15.129:0/4207178771 conn(0x7f2fd80fd8c0 msgr2=0x7f2fd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 -- 172.21.15.129:0/4207178771 shutdown_connections 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.722+0000 7f2fdfe50640 1 -- 172.21.15.129:0/4207178771 wait complete. 2024-09-05T01:32:47.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 Processor -- start 2024-09-05T01:32:47.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 -- start start 2024-09-05T01:32:47.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:47.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8102790 0x7f2fd819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:47.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8106f10 0x7f2fd819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:47.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8111e20 con 0x7f2fd8101de0 2024-09-05T01:32:47.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8111ca0 con 0x7f2fd8102790 2024-09-05T01:32:47.720 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdfe50640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd8111fa0 con 0x7f2fd8106f10 2024-09-05T01:32:47.720 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:47.721 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:32:47.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:47.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51150/0 (socket says 172.21.15.129:51150) 2024-09-05T01:32:47.721 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 -- 172.21.15.129:0/1083022528 learned_addr learned my addr 172.21.15.129:0/1083022528 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:47.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fdd3eb640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8102790 0x7f2fd819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:47.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fde3ed640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8106f10 0x7f2fd819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:47.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 -- 172.21.15.129:0/1083022528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8106f10 msgr2=0x7f2fd819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:47.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8106f10 0x7f2fd819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.722 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 -- 172.21.15.129:0/1083022528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8102790 msgr2=0x7f2fd819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:47.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8102790 0x7f2fd819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.723+0000 7f2fddbec640 1 -- 172.21.15.129:0/1083022528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fd819f5c0 con 0x7f2fd8101de0 2024-09-05T01:32:47.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fddbec640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400e9f0 tx=0x7f2fd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:47.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400cd80 con 0x7f2fd8101de0 2024-09-05T01:32:47.723 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fd40054d0 con 0x7f2fd8101de0 2024-09-05T01:32:47.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4010490 con 0x7f2fd8101de0 2024-09-05T01:32:47.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd819f8b0 con 0x7f2fd8101de0 2024-09-05T01:32:47.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.724+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd81a7190 con 0x7f2fd8101de0 2024-09-05T01:32:47.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.725+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd81021c0 con 0x7f2fd8101de0 2024-09-05T01:32:47.724 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.726+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f2fd4005000 con 0x7f2fd8101de0 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.727+0000 7f2fceffd640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 0x7f2fc40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.727+0000 7f2fdd3eb640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 0x7f2fc40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.727+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f2fd40cfae0 con 0x7f2fd8101de0 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.728+0000 7f2fdd3eb640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 0x7f2fc40a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8031440 tx=0x7f2fc80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.730+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fd40541b0 con 0x7f2fd8101de0 2024-09-05T01:32:47.725 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.933+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fd819b970 con 0x7f2fd8101de0 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.934+0000 7f2fceffd640 1 -- 172.21.15.129:0/1083022528 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f2fd819b970 con 0x7f2fd8101de0 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 msgr2=0x7f2fc40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 0x7f2fc40a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8031440 tx=0x7f2fc80095f0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 msgr2=0x7f2fd819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:47.726 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400e9f0 tx=0x7f2fd400edf0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 shutdown_connections 2024-09-05T01:32:47.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f2fc40a2c00 0x7f2fc40a50c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f2fd8106f10 0x7f2fd819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fd8102790 0x7f2fd819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.727 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 --2- 172.21.15.129:0/1083022528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2fd8101de0 0x7f2fd819a610 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:47.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 >> 172.21.15.129:0/1083022528 conn(0x7f2fd80fd8c0 msgr2=0x7f2fd80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:47.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 shutdown_connections 2024-09-05T01:32:47.728 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:46.936+0000 7f2fdfe50640 1 -- 172.21.15.129:0/1083022528 wait complete. 2024-09-05T01:32:48.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:48 smithi183 ceph-mon[34589]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.410+0000 7fa7f6082640 1 -- 172.21.15.129:0/322383676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 msgr2=0x7fa7f01020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.410+0000 7fa7f6082640 1 --2- 172.21.15.129:0/322383676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f01020d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009900 tx=0x7fa7e402a410 comp rx=0 tx=0).stop 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 -- 172.21.15.129:0/322383676 shutdown_connections 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 --2- 172.21.15.129:0/322383676 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 0x7fa7f0108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 --2- 172.21.15.129:0/322383676 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f01020d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 --2- 172.21.15.129:0/322383676 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 0x7fa7f0100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 -- 172.21.15.129:0/322383676 >> 172.21.15.129:0/322383676 conn(0x7fa7f00fc1e0 msgr2=0x7fa7f00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 -- 172.21.15.129:0/322383676 shutdown_connections 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 -- 172.21.15.129:0/322383676 wait complete. 2024-09-05T01:32:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 Processor -- start 2024-09-05T01:32:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.411+0000 7fa7f6082640 1 -- start start 2024-09-05T01:32:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 0x7fa7f010b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.415 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 0x7fa7f0109f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d740 con 0x7fa7f0101c50 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d5c0 con 0x7fa7f0100a60 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7f6082640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010d8c0 con 0x7fa7f0102610 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51162/0 (socket says 172.21.15.129:51162) 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 -- 172.21.15.129:0/4001166959 learned_addr learned my addr 172.21.15.129:0/4001166959 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 0x7fa7f010b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7effff640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 0x7fa7f0109f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 -- 172.21.15.129:0/4001166959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 msgr2=0x7fa7f0109f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 0x7fa7f0109f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 -- 172.21.15.129:0/4001166959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 msgr2=0x7fa7f010b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 0x7fa7f010b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 -- 172.21.15.129:0/4001166959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f010a7f0 con 0x7fa7f0101c50 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40314e0 tx=0x7fa7e4003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7ecff9640 1 -- 172.21.15.129:0/4001166959 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e403f070 con 0x7fa7f0101c50 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.412+0000 7fa7ecff9640 1 -- 172.21.15.129:0/4001166959 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e4031e40 con 0x7fa7f0101c50 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.413+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f01a1d30 con 0x7fa7f0101c50 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.413+0000 7fa7ecff9640 1 -- 172.21.15.129:0/4001166959 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4035030 con 0x7fa7f0101c50 2024-09-05T01:32:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.413+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f01a21d0 con 0x7fa7f0101c50 2024-09-05T01:32:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7ecff9640 1 -- 172.21.15.129:0/4001166959 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa7e4042070 con 0x7fa7f0101c50 2024-09-05T01:32:48.423 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 Processor -- start 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 -- start start 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18209a0 0x5580c1820da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18212e0 0x5580c1825da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580c17842f0 con 0x5580c1826370 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580c172c270 con 0x5580c18212e0 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5580c17802b0 con 0x5580c18209a0 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.415+0000 7fa7ecff9640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 0x7fa7d80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51176/0 (socket says 172.21.15.129:51176) 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 -- 172.21.15.129:0/1189066417 learned_addr learned my addr 172.21.15.129:0/1189066417 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7effff640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 0x7fa7d80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.424 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18212e0 0x5580c1825da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7ecff9640 1 -- 172.21.15.129:0/4001166959 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7e40efcf0 con 0x7fa7f0101c50 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7effff640 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18209a0 0x5580c1820da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 -- 172.21.15.129:0/1189066417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18209a0 msgr2=0x5580c1820da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18209a0 0x5580c1820da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 -- 172.21.15.129:0/1189066417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18212e0 msgr2=0x5580c1825da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18212e0 0x5580c1825da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 -- 172.21.15.129:0/1189066417 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c172b340 con 0x5580c1826370 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7effff640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 0x7fa7d80a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8005fd0 tx=0x7fa7e8009b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40bc180 tx=0x7fa7e408f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e070005030a59f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7d5ffb640 1 -- 172.21.15.129:0/1189066417 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4049070 con 0x5580c1826370 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7d5ffb640 1 -- 172.21.15.129:0/1189066417 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e40bc1f0 con 0x5580c1826370 2024-09-05T01:32:48.425 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.416+0000 7fa7d5ffb640 1 -- 172.21.15.129:0/1189066417 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40b8860 con 0x5580c1826370 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- 172.21.15.129:0/1189066417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 msgr2=0x5580c1828830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40bc180 tx=0x7fa7e408f2b0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- 172.21.15.129:0/1189066417 shutdown_connections 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c1826370 0x5580c1828830 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18212e0 0x5580c1825da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/1189066417 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18209a0 0x5580c1820da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- 172.21.15.129:0/1189066417 >> 172.21.15.129:0/1189066417 conn(0x5580c181c010 msgr2=0x5580c181c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- 172.21.15.129:0/1189066417 shutdown_connections 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- 172.21.15.129:0/1189066417 wait complete. 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.417+0000 7fa7f8e20740 1 -- start start 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 Processor -- start 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.426 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c182ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 0x5580c1831ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580c1829100 con 0x5580c18209a0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5580c170d290 con 0x5580c1826370 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x5580c1829280 con 0x5580c18212e0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7f8e20740 10 client.0 _ll_register_callbacks cb 0x5580c173e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51188/0 (socket says 172.21.15.129:51188) 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/3328823478 learned_addr learned my addr 172.21.15.129:0/3328823478 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c182ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7effff640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 0x5580c1831ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/3328823478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 msgr2=0x5580c182ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c182ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.427 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 msgr2=0x5580c1831ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 0x5580c1831ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/3328823478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c1497210 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.418+0000 7fa7effff640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 0x5580c1831ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0010840 tx=0x7fa7e0010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38606f74a4e9d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e000fc50 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7e000fdf0 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e40bc1f0 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7e0024050 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 10 client.17223 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5580c1563620 con 0x5580c18209a0 2024-09-05T01:32:48.428 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e40bc1f0 con 0x5580c18209a0 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e000e540 con 0x5580c18209a0 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 10 client.17223 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 10 client.17223 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 20 client.17223 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1836690 0x5580c1838b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 4 client.17223 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.419+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580c1791a20 con 0x5580c1836690 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.420+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1836690 0x5580c1838b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.420+0000 7fa7ceffd640 10 client.17223 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.420+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1836690 0x5580c1838b50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8aa8917457eff5 server_cookie=d6c05cf3032ea403 in_seq=0 out_seq=0 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3328823478 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580c1791a20 con 0x5580c1836690 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7ceffd640 10 client.17223 handle_command_reply: tid=0 2024-09-05T01:32:48.429 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 client.17223 shutdown 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 msgr2=0x5580c182a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0010840 tx=0x7fa7e0010d00 comp rx=0 tx=0).stop 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 shutdown_connections 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1836690 0x5580c1838b50 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c1826370 0x5580c1831ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c182ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7ceffd640 0 client.17223 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3328823478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182a940 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 >> 172.21.15.129:0/3328823478 conn(0x5580c181c010 msgr2=0x5580c181d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 shutdown_connections 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3328823478 wait complete. 2024-09-05T01:32:48.430 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.421+0000 7fa7f8e20740 20 client.17223 trim_cache size 0 max 16384 2024-09-05T01:32:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.435+0000 7fa7f8e20740 1 Processor -- start 2024-09-05T01:32:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 -- start start 2024-09-05T01:32:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 0x5580c1831de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.442 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c181d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01af250 con 0x5580c181c010 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01acb20 con 0x5580c18209a0 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01aa970 con 0x5580c18212e0 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51194/0 (socket says 172.21.15.129:51194) 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/2342795510 learned_addr learned my addr 172.21.15.129:0/2342795510 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 0x5580c1831de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/2342795510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 msgr2=0x5580c181d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c181d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/2342795510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 msgr2=0x5580c1831de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 0x5580c1831de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.443 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 -- 172.21.15.129:0/2342795510 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580c18217c0 con 0x5580c181c010 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 0x5580c1831de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0010810 tx=0x7fa7e000a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb5dad6d98d6392e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7d57fa640 1 -- 172.21.15.129:0/2342795510 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e002d070 con 0x5580c181c010 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7d57fa640 1 -- 172.21.15.129:0/2342795510 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e000fc50 con 0x5580c181c010 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.436+0000 7fa7d57fa640 1 -- 172.21.15.129:0/2342795510 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00054d0 con 0x5580c181c010 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- 172.21.15.129:0/2342795510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 msgr2=0x5580c181d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0010810 tx=0x7fa7e000a6e0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- 172.21.15.129:0/2342795510 shutdown_connections 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c181d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c18209a0 0x5580c1831de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/2342795510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c181c010 0x5580c181d400 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- 172.21.15.129:0/2342795510 >> 172.21.15.129:0/2342795510 conn(0x5580c1825ec0 msgr2=0x5580c181b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.444 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- 172.21.15.129:0/2342795510 shutdown_connections 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- 172.21.15.129:0/2342795510 wait complete. 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.437+0000 7fa7f8e20740 1 -- start start 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 Processor -- start 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c181c010 0x5580c182f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c1830bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01aafe0 con 0x5580c18209a0 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01ac450 con 0x5580c181c010 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f01ab6b0 con 0x5580c18212e0 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7f8e20740 10 client.0 _ll_register_callbacks cb 0x7fa7e000a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51198/0 (socket says 172.21.15.129:51198) 2024-09-05T01:32:48.445 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 -- 172.21.15.129:0/3916783526 learned_addr learned my addr 172.21.15.129:0/3916783526 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c181c010 0x5580c182f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7effff640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c1830bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 -- 172.21.15.129:0/3916783526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 msgr2=0x5580c1830bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c1830bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 -- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c181c010 msgr2=0x5580c182f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c181c010 0x5580c182f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.438+0000 7fa7eeffd640 1 -- 172.21.15.129:0/3916783526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580c18311c0 con 0x5580c18209a0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7eeffd640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40b8bc0 tx=0x7fa7e4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6910647dc2625ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e40d5060 con 0x5580c18209a0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa7e40c0170 con 0x5580c18209a0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7e000fc50 con 0x5580c18209a0 2024-09-05T01:32:48.446 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa7e4049040 con 0x5580c18209a0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 10 client.17229 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5580c18a1510 con 0x5580c18209a0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7e000fc50 con 0x5580c18209a0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e40dc0b0 con 0x5580c18209a0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 10 client.17229 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 10 client.17229 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 20 client.17229 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1826370 0x5580c18a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 4 client.17229 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.439+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580c178bd70 con 0x5580c1826370 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.440+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1826370 0x5580c18a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.440+0000 7fa7ef7fe640 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1826370 0x5580c18a3da0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d01bca7a60b9481 server_cookie=5f11dcf39f3c22a6 in_seq=0 out_seq=0 2024-09-05T01:32:48.447 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.440+0000 7fa7ceffd640 10 client.17229 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:48.473 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:48 smithi129 ceph-mon[30979]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-09-05T01:32:48.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:48 smithi194 ceph-mon[34941]: pgmap v877: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-09-05T01:32:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.508+0000 7fa7ceffd640 1 -- 172.21.15.129:0/3916783526 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580c178bd70 con 0x5580c1826370 2024-09-05T01:32:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7ceffd640 10 client.17229 handle_command_reply: tid=0 2024-09-05T01:32:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 client.17229 shutdown 2024-09-05T01:32:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 msgr2=0x5580c182fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40b8bc0 tx=0x7fa7e4009260 comp rx=0 tx=0).stop 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 shutdown_connections 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x5580c1826370 0x5580c18a3da0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x5580c18212e0 0x5580c1830bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580c18209a0 0x5580c182fba0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 --2- 172.21.15.129:0/3916783526 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5580c181c010 0x5580c182f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 >> 172.21.15.129:0/3916783526 conn(0x5580c1825ec0 msgr2=0x5580c181c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7ceffd640 0 client.17229 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 shutdown_connections 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 1 -- 172.21.15.129:0/3916783526 wait complete. 2024-09-05T01:32:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.509+0000 7fa7f8e20740 20 client.17229 trim_cache size 0 max 16384 2024-09-05T01:32:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:{ 2024-09-05T01:32:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "message": "", 2024-09-05T01:32:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout: "return_code": 0 2024-09-05T01:32:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stdout:} 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 msgr2=0x7fa7d80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 0x7fa7d80a50f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8005fd0 tx=0x7fa7e8009b50 comp rx=0 tx=0).stop 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 msgr2=0x7fa7f0109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40314e0 tx=0x7fa7e4003c80 comp rx=0 tx=0).stop 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 shutdown_connections 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7d80a2c30 0x7fa7d80a50f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa7f0102610 0x7fa7f0109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0101c50 0x7fa7f0109a20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 --2- 172.21.15.129:0/4001166959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa7f0100a60 0x7fa7f010b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 >> 172.21.15.129:0/4001166959 conn(0x7fa7f00fc1e0 msgr2=0x7fa7f0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 shutdown_connections 2024-09-05T01:32:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi129.front.sepia.ceph.com.smithi129.stderr:2024-09-05T01:32:48.510+0000 7fa7f6082640 1 -- 172.21.15.129:0/4001166959 wait complete. 2024-09-05T01:32:48.530 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24488 quiesce db --cancel --all' 2024-09-05T01:32:48.531 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24488 quiesce db --cancel --all 2024-09-05T01:32:48.777 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:49.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- 172.21.15.129:0/2158534216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb208103760 msgr2=0x7fb208107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 --2- 172.21.15.129:0/2158534216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb208103760 0x7fb208107ba0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009930 tx=0x7fb1f002a420 comp rx=0 tx=0).stop 2024-09-05T01:32:49.261 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- 172.21.15.129:0/2158534216 shutdown_connections 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 --2- 172.21.15.129:0/2158534216 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb2081082e0 0x7fb20810ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 --2- 172.21.15.129:0/2158534216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb208103760 0x7fb208107ba0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 --2- 172.21.15.129:0/2158534216 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208102db0 0x7fb208103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- 172.21.15.129:0/2158534216 >> 172.21.15.129:0/2158534216 conn(0x7fb2080fc7d0 msgr2=0x7fb2080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- 172.21.15.129:0/2158534216 shutdown_connections 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- 172.21.15.129:0/2158534216 wait complete. 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 Processor -- start 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.244+0000 7fb20ec33640 1 -- start start 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 0x7fb208198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 0x7fb208198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb208111f20 con 0x7fb2081082e0 2024-09-05T01:32:49.262 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb208111da0 con 0x7fb208103760 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081120a0 con 0x7fb208102db0 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51204/0 (socket says 172.21.15.129:51204) 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 -- 172.21.15.129:0/512116451 learned_addr learned my addr 172.21.15.129:0/512116451 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 0x7fb208198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb207fff640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 0x7fb208198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 -- 172.21.15.129:0/512116451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 msgr2=0x7fb208198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 0x7fb208198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 -- 172.21.15.129:0/512116451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 msgr2=0x7fb208198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 0x7fb208198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 -- 172.21.15.129:0/512116451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20819d4b0 con 0x7fb2081082e0 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb207fff640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 0x7fb208198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:49.263 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 0x7fb208198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20d1d0640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb20000d750 tx=0x7fb20000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb205ffb640 1 -- 172.21.15.129:0/512116451 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200017070 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb205ffb640 1 -- 172.21.15.129:0/512116451 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2000054d0 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20819d7a0 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.245+0000 7fb205ffb640 1 -- 172.21.15.129:0/512116451 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200004c60 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.246+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a5080 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.248+0000 7fb205ffb640 1 -- 172.21.15.129:0/512116451 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb200004e00 con 0x7fb2081082e0 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 Processor -- start 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 -- start start 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.264 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96677ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9667a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561d965cd200 con 0x561d966688e0 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d96573dc0 con 0x561d96669200 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb21100c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561d965c91c0 con 0x561d96677ff0 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51206/0 (socket says 172.21.15.129:51206) 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 -- 172.21.15.129:0/1903579516 learned_addr learned my addr 172.21.15.129:0/1903579516 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96677ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb205ffb640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 -- 172.21.15.129:0/1903579516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 msgr2=0x561d9667a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.265 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9667a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 -- 172.21.15.129:0/1903579516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 msgr2=0x561d96677ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96677ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb207fff640 1 -- 172.21.15.129:0/1903579516 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d9633e620 con 0x561d966688e0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb20d1d0640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.249+0000 7fb205ffb640 1 -- 172.21.15.129:0/512116451 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb2000cedf0 con 0x7fb2081082e0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.250+0000 7fb207fff640 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0006010 tx=0x7fb1f0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece318af2f4a74b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.250+0000 7fb1feffd640 1 -- 172.21.15.129:0/1903579516 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0035050 con 0x561d966688e0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.250+0000 7fb1feffd640 1 -- 172.21.15.129:0/1903579516 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f0031630 con 0x561d966688e0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.250+0000 7fb1feffd640 1 -- 172.21.15.129:0/1903579516 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0005640 con 0x561d966688e0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.250+0000 7fb20d1d0640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb20009b1d0 tx=0x7fb2000bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.266 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- 172.21.15.129:0/1903579516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 msgr2=0x561d96668cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0006010 tx=0x7fb1f0005e50 comp rx=0 tx=0).stop 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- 172.21.15.129:0/1903579516 shutdown_connections 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9667a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96677ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- 172.21.15.129:0/1903579516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96668cc0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- 172.21.15.129:0/1903579516 >> 172.21.15.129:0/1903579516 conn(0x561d96664f90 msgr2=0x561d96665370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- 172.21.15.129:0/1903579516 shutdown_connections 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- 172.21.15.129:0/1903579516 wait complete. 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- start start 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 Processor -- start 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d96674c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.267 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d96675170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561d9667abc0 con 0x561d966688e0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561d96567c50 con 0x561d96677ff0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.251+0000 7fb21100c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x561d9667ad40 con 0x561d96669200 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb21100c740 10 client.0 _ll_register_callbacks cb 0x561d96579ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51214/0 (socket says 172.21.15.129:51214) 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 -- 172.21.15.129:0/524478742 learned_addr learned my addr 172.21.15.129:0/524478742 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d96674c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d96675170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 -- 172.21.15.129:0/524478742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 msgr2=0x561d96674c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d96674c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 -- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 msgr2=0x561d96675170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d96675170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 -- 172.21.15.129:0/524478742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d966759a0 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb207fff640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb1f002a460 tx=0x7fb1f0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8e5ef447bf4f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0035030 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f0031630 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb1f0004440 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb1f005a050 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.252+0000 7fb21100c740 10 client.17241 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561d966867b0 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f0031630 con 0x561d966688e0 2024-09-05T01:32:49.269 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb1f003c710 con 0x561d966688e0 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 10 client.17241 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 10 client.17241 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 20 client.17241 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d96687b00 0x561d96689ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 4 client.17241 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d965da770 con 0x561d96687b00 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.253+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d96687b00 0x561d96689ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb1e7fff640 10 client.17241 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:49.270 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d96687b00 0x561d96689ef0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d435f21ef2a52af server_cookie=d2496623587f18f4 in_seq=0 out_seq=0 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb1e7fff640 1 -- 172.21.15.129:0/524478742 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d965da770 con 0x561d96687b00 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb1e7fff640 10 client.17241 handle_command_reply: tid=0 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 client.17241 shutdown 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 msgr2=0x561d96677910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb1f002a460 tx=0x7fb1f0005d50 comp rx=0 tx=0).stop 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 shutdown_connections 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d96687b00 0x561d96689ef0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d96675170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d96674c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 --2- 172.21.15.129:0/524478742 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96677910 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 >> 172.21.15.129:0/524478742 conn(0x561d96664f90 msgr2=0x561d9666acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.254+0000 7fb1e7fff640 0 client.17241 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:49.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.255+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 shutdown_connections 2024-09-05T01:32:49.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.255+0000 7fb21100c740 1 -- 172.21.15.129:0/524478742 wait complete. 2024-09-05T01:32:49.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.255+0000 7fb21100c740 20 client.17241 trim_cache size 0 max 16384 2024-09-05T01:32:49.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 Processor -- start 2024-09-05T01:32:49.274 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 -- start start 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96673bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9668f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081af450 con 0x561d966688e0 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081acd20 con 0x561d96669200 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb21100c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081aab70 con 0x561d96677ff0 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51218/0 (socket says 172.21.15.129:51218) 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 -- 172.21.15.129:0/595502558 learned_addr learned my addr 172.21.15.129:0/595502558 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 -- 172.21.15.129:0/595502558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 msgr2=0x561d9668f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9668f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.275 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 -- 172.21.15.129:0/595502558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 msgr2=0x561d96673bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96673bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96673bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 -- 172.21.15.129:0/595502558 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d96660730 con 0x561d966688e0 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.272+0000 7fb207fff640 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009930 tx=0x7fb1f0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caa86538c07edb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb1fe7fc640 1 -- 172.21.15.129:0/595502558 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0035070 con 0x561d966688e0 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb1fe7fc640 1 -- 172.21.15.129:0/595502558 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f004e040 con 0x561d966688e0 2024-09-05T01:32:49.276 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb1fe7fc640 1 -- 172.21.15.129:0/595502558 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00035f0 con 0x561d966688e0 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 -- 172.21.15.129:0/595502558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 msgr2=0x561d96683e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009930 tx=0x7fb1f0005bf0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 -- 172.21.15.129:0/595502558 shutdown_connections 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96677ff0 0x561d9668f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96669200 0x561d96673bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 --2- 172.21.15.129:0/595502558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d96683e60 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 -- 172.21.15.129:0/595502558 >> 172.21.15.129:0/595502558 conn(0x561d96664f90 msgr2=0x561d966734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.280 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.273+0000 7fb21100c740 1 -- 172.21.15.129:0/595502558 shutdown_connections 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 -- 172.21.15.129:0/595502558 wait complete. 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 -- start start 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 Processor -- start 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d9667b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d9667ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ab1e0 con 0x561d966688e0 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ac650 con 0x561d96677ff0 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081ab8b0 con 0x561d96669200 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb21100c740 10 client.0 _ll_register_callbacks cb 0x7fb1f8009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb207fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51226/0 (socket says 172.21.15.129:51226) 2024-09-05T01:32:49.281 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb207fff640 1 -- 172.21.15.129:0/4085877415 learned_addr learned my addr 172.21.15.129:0/4085877415 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.274+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d9667b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d9667ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 -- 172.21.15.129:0/4085877415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 msgr2=0x561d9667b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d9667b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 -- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 msgr2=0x561d9667ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d9667ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 -- 172.21.15.129:0/4085877415 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d966f0550 con 0x561d966688e0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb207fff640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00096c0 tx=0x7fb1f0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37db4ffe5f5517a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb1e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0035040 con 0x561d966688e0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mon.0 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb1f00315e0 con 0x561d966688e0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f004e040 con 0x561d966688e0 2024-09-05T01:32:49.282 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb1f0071050 con 0x561d966688e0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb21100c740 10 client.17247 fetch_fsmap learned FSMap version 34 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561d966f1830 con 0x561d966688e0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.275+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mon.0 v2:172.21.15.129:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f004e040 con 0x561d966688e0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mon.0 v2:172.21.15.129:3300/0 5 ==== fsmap(e 34) ==== 8354+0+0 (secure 0 0 0) 0x7fb1f003a510 con 0x561d966688e0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 10 client.17247 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 10 client.17247 resolve_mds: validated gid 24488 aka daemon mds.b 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 20 client.17247 populate_metadata read hostname 'smithi129' 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d966f1f50 0x561d966f4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 4 client.17247 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 --> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x561d965d4c80 con 0x561d966f1f50 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d966f1f50 0x561d966f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb20c9cf640 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d966f1f50 0x561d966f4370 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f86d0fdb9f6e35 server_cookie=a6b31bd65707460a in_seq=0 out_seq=0 2024-09-05T01:32:49.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.276+0000 7fb1e7fff640 10 client.17247 ms_handle_connect on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb1e7fff640 1 -- 172.21.15.129:0/4085877415 <== mds.0 v2:172.21.15.183:6838/3533858614 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x561d965d4c80 con 0x561d966f1f50 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb1e7fff640 10 client.17247 handle_command_reply: tid=0 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 client.17247 shutdown 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 msgr2=0x561d9667af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00096c0 tx=0x7fb1f0005d50 comp rx=0 tx=0).stop 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 shutdown_connections 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:6838/3533858614,v1:172.21.15.183:6839/3533858614] conn(0x561d966f1f50 0x561d966f4370 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561d96677ff0 0x561d9667ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x561d96669200 0x561d9667b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 --2- 172.21.15.129:0/4085877415 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d966688e0 0x561d9667af90 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 >> 172.21.15.129:0/4085877415 conn(0x561d96664f90 msgr2=0x561d96673d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb1e7fff640 0 client.17247 ms_handle_reset on v2:172.21.15.183:6838/3533858614 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 shutdown_connections 2024-09-05T01:32:49.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 1 -- 172.21.15.129:0/4085877415 wait complete. 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.277+0000 7fb21100c740 20 client.17247 trim_cache size 0 max 16384 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 34, 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "leader": 24488, 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "set_version": 68, 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout: "sets": {} 2024-09-05T01:32:49.285 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 msgr2=0x7fb1f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb20009b1d0 tx=0x7fb2000bb040 comp rx=0 tx=0).stop 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 msgr2=0x7fb20819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb20000d750 tx=0x7fb20000dc10 comp rx=0 tx=0).stop 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 shutdown_connections 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb1f40a2c00 0x7fb1f40a50c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb2081082e0 0x7fb20819cdd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb208103760 0x7fb208198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 --2- 172.21.15.129:0/512116451 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb208102db0 0x7fb208198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:49.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 >> 172.21.15.129:0/512116451 conn(0x7fb2080fc7d0 msgr2=0x7fb208105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:49.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 shutdown_connections 2024-09-05T01:32:49.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:49.278+0000 7fb20ec33640 1 -- 172.21.15.129:0/512116451 wait complete. 2024-09-05T01:32:49.986 INFO:tasks.quiescer:done joining Quiescers 2024-09-05T01:32:49.986 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-05T01:32:49.996 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:32:50.004 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-05T01:32:50.014 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T01:32:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- 172.21.15.129:0/167847010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c101b30 msgr2=0x7f226c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 --2- 172.21.15.129:0/167847010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c101b30 0x7f226c101f30 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2264009900 tx=0x7f226402a410 comp rx=0 tx=0).stop 2024-09-05T01:32:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- 172.21.15.129:0/167847010 shutdown_connections 2024-09-05T01:32:50.163 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 --2- 172.21.15.129:0/167847010 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c1036e0 0x7f226c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 --2- 172.21.15.129:0/167847010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 --2- 172.21.15.129:0/167847010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c101b30 0x7f226c101f30 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- 172.21.15.129:0/167847010 >> 172.21.15.129:0/167847010 conn(0x7f226c0fd2b0 msgr2=0x7f226c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- 172.21.15.129:0/167847010 shutdown_connections 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- 172.21.15.129:0/167847010 wait complete. 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 Processor -- start 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.158+0000 7f2270865640 1 -- start start 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c101b30 0x7f226c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10d640 con 0x7f226c1036e0 2024-09-05T01:32:50.164 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10d4c0 con 0x7f226c102d20 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f226c10d7c0 con 0x7f226c101b30 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51240/0 (socket says 172.21.15.129:51240) 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 -- 172.21.15.129:0/3825281944 learned_addr learned my addr 172.21.15.129:0/3825281944 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2269d9b640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226a59c640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c101b30 0x7f226c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 -- 172.21.15.129:0/3825281944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c101b30 msgr2=0x7f226c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c101b30 0x7f226c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 -- 172.21.15.129:0/3825281944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 msgr2=0x7f226c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 -- 172.21.15.129:0/3825281944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f226c19f980 con 0x7f226c1036e0 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2269d9b640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f226ad9d640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c970 tx=0x7f225c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.165 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c007bb0 con 0x7f226c1036e0 2024-09-05T01:32:50.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f225c007d50 con 0x7f226c1036e0 2024-09-05T01:32:50.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f226c19fc70 con 0x7f226c1036e0 2024-09-05T01:32:50.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c004e10 con 0x7f226c1036e0 2024-09-05T01:32:50.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.159+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f226c1a0170 con 0x7f226c1036e0 2024-09-05T01:32:50.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.161+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f226c108f10 con 0x7f226c1036e0 2024-09-05T01:32:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.162+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f225c005000 con 0x7f226c1036e0 2024-09-05T01:32:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.162+0000 7f225b7fe640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 0x7f22540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.163+0000 7f226a59c640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 0x7f22540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.163+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f225c0cf590 con 0x7f226c1036e0 2024-09-05T01:32:50.167 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.163+0000 7f226a59c640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 0x7f22540a5050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f22640314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.168 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.166+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f225c014030 con 0x7f226c1036e0 2024-09-05T01:32:50.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:50 smithi129 ceph-mon[30979]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.0 MiB/s wr, 769 op/s 2024-09-05T01:32:50.338 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.336+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f226c02da10 con 0x7f226c1036e0 2024-09-05T01:32:50.339 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.337+0000 7f225b7fe640 1 -- 172.21.15.129:0/3825281944 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+25009 (secure 0 0 0) 0x7f225c09c0e0 con 0x7f226c1036e0 2024-09-05T01:32:50.340 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:50.340 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":93,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:26:34.410093+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:32:50.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 msgr2=0x7f22540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.341 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 0x7f22540a5050 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2264000c00 tx=0x7f22640314a0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 msgr2=0x7f226c19f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f225c00c970 tx=0x7f225c00ce30 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 shutdown_connections 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f22540a2b90 0x7f22540a5050 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f226c1036e0 0x7f226c19f310 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.339+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f226c102d20 0x7f226c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.340+0000 7f2270865640 1 --2- 172.21.15.129:0/3825281944 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f226c101b30 0x7f226c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.340+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 >> 172.21.15.129:0/3825281944 conn(0x7f226c0fd2b0 msgr2=0x7f226c1060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.340+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 shutdown_connections 2024-09-05T01:32:50.342 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.340+0000 7f2270865640 1 -- 172.21.15.129:0/3825281944 wait complete. 2024-09-05T01:32:50.353 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T01:08:27.962073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T01:13:02.228925+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T01:13:04.722842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T01:13:11.133644+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T01:32:50.354 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T01:32:50.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:50 smithi183 ceph-mon[34589]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.0 MiB/s wr, 769 op/s 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/2689271670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c4100a00 msgr2=0x7ff1c4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/2689271670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c4100a00 0x7ff1c4100e00 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009900 tx=0x7ff1b402a410 comp rx=0 tx=0).stop 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/2689271670 shutdown_connections 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/2689271670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c41025b0 0x7ff1c4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/2689271670 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/2689271670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c4100a00 0x7ff1c4100e00 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/2689271670 >> 172.21.15.129:0/2689271670 conn(0x7ff1c40fc140 msgr2=0x7ff1c40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/2689271670 shutdown_connections 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/2689271670 wait complete. 2024-09-05T01:32:50.459 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 Processor -- start 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- start start 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 0x7ff1c4196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c41971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c590 con 0x7ff1c41025b0 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c410 con 0x7ff1c4100a00 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.455+0000 7ff1ca5b7640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c410c710 con 0x7ff1c4101bf0 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51262/0 (socket says 172.21.15.129:51262) 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 -- 172.21.15.129:0/1756576076 learned_addr learned my addr 172.21.15.129:0/1756576076 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1bbfff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c41971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c3fff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 0x7ff1c4196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.460 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 -- 172.21.15.129:0/1756576076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 msgr2=0x7ff1c41971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c41971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 -- 172.21.15.129:0/1756576076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 msgr2=0x7ff1c4196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 0x7ff1c4196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 -- 172.21.15.129:0/1756576076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c419e880 con 0x7ff1c41025b0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c3fff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 0x7ff1c4196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1bbfff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c8b54640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b970 tx=0x7ff1bc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc019070 con 0x7ff1c41025b0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1bc014440 con 0x7ff1c41025b0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c419eb70 con 0x7ff1c41025b0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0075d0 con 0x7ff1c41025b0 2024-09-05T01:32:50.461 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.456+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c419f070 con 0x7ff1c41025b0 2024-09-05T01:32:50.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.457+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c4107e90 con 0x7ff1c41025b0 2024-09-05T01:32:50.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.459+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7ff1bc01c070 con 0x7ff1c41025b0 2024-09-05T01:32:50.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.460+0000 7ff1c1ffb640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 0x7ff1b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.460+0000 7ff1c3fff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 0x7ff1b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.462 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.460+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7ff1bc0cf010 con 0x7ff1c41025b0 2024-09-05T01:32:50.463 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.460+0000 7ff1c3fff640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 0x7ff1b00a5050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006010 tx=0x7ff1b4031d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.463+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1bc09bb60 con 0x7ff1c41025b0 2024-09-05T01:32:50.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:50 smithi194 ceph-mon[34941]: pgmap v878: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.0 MiB/s wr, 769 op/s 2024-09-05T01:32:50.630 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.628+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff1c402d7c0 con 0x7ff1c41025b0 2024-09-05T01:32:50.631 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.629+0000 7ff1c1ffb640 1 -- 172.21.15.129:0/1756576076 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7ff1bc09b460 con 0x7ff1c41025b0 2024-09-05T01:32:50.631 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 1 2024-09-05T01:32:50.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 msgr2=0x7ff1b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 0x7ff1b00a5050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4006010 tx=0x7ff1b4031d10 comp rx=0 tx=0).stop 2024-09-05T01:32:50.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 msgr2=0x7ff1c419e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.633 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00b970 tx=0x7ff1bc00be30 comp rx=0 tx=0).stop 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 shutdown_connections 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7ff1b00a2b90 0x7ff1b00a5050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff1c41025b0 0x7ff1c419e1e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7ff1c4101bf0 0x7ff1c41971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 --2- 172.21.15.129:0/1756576076 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff1c4100a00 0x7ff1c4196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.634 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 >> 172.21.15.129:0/1756576076 conn(0x7ff1c40fc140 msgr2=0x7ff1c4105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 shutdown_connections 2024-09-05T01:32:50.635 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.632+0000 7ff1ca5b7640 1 -- 172.21.15.129:0/1756576076 wait complete. 2024-09-05T01:32:50.644 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T01:32:50.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- 172.21.15.129:0/2868112764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed41025b0 msgr2=0x7f6ed4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- 172.21.15.129:0/2868112764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed41025b0 0x7f6ed4108e40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009ca0 tx=0x7f6ecc02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- 172.21.15.129:0/2868112764 shutdown_connections 2024-09-05T01:32:50.750 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- 172.21.15.129:0/2868112764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed41025b0 0x7f6ed4108e40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- 172.21.15.129:0/2868112764 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- 172.21.15.129:0/2868112764 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed4100a00 0x7f6ed4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- 172.21.15.129:0/2868112764 >> 172.21.15.129:0/2868112764 conn(0x7f6ed40fc140 msgr2=0x7f6ed40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- 172.21.15.129:0/2868112764 shutdown_connections 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- 172.21.15.129:0/2868112764 wait complete. 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 Processor -- start 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- start start 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed41025b0 0x7f6ed419e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410c530 con 0x7f6ed4100a00 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410c3b0 con 0x7f6ed4101bf0 2024-09-05T01:32:50.751 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.746+0000 7f6edabd6640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed410c6b0 con 0x7f6ed41025b0 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51286/0 (socket says 172.21.15.129:51286) 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 -- 172.21.15.129:0/1862064457 learned_addr learned my addr 172.21.15.129:0/1862064457 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed9173640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed41025b0 0x7f6ed419e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed3fff640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 -- 172.21.15.129:0/1862064457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed41025b0 msgr2=0x7f6ed419e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed41025b0 0x7f6ed419e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 -- 172.21.15.129:0/1862064457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 msgr2=0x7f6ed41970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed41970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 -- 172.21.15.129:0/1862064457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed419e7b0 con 0x7f6ed4100a00 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed3fff640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed41970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:32:50.752 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed8972640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800d950 tx=0x7f6ec800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8017070 con 0x7f6ed4100a00 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ec8009c30 con 0x7f6ed4100a00 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800f620 con 0x7f6ed4100a00 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed419eaa0 con 0x7f6ed4100a00 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.747+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed419efa0 con 0x7f6ed4100a00 2024-09-05T01:32:50.753 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.748+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed4107e10 con 0x7f6ed4100a00 2024-09-05T01:32:50.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.750+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f6ec801a070 con 0x7f6ed4100a00 2024-09-05T01:32:50.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.751+0000 7f6ed1ffb640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 0x7f6ec40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:50.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.751+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f6ec80cef60 con 0x7f6ed4100a00 2024-09-05T01:32:50.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.751+0000 7f6ed3fff640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 0x7f6ec40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:50.754 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.752+0000 7f6ed3fff640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 0x7f6ec40a5050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ed41980a0 tx=0x7f6ec0005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:50.756 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.754+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ec8009de0 con 0x7f6ed4100a00 2024-09-05T01:32:50.921 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.921+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6ed402d7c0 con 0x7f6ed4100a00 2024-09-05T01:32:50.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.921+0000 7f6ed1ffb640 1 -- 172.21.15.129:0/1862064457 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f6ec809bab0 con 0x7f6ed4100a00 2024-09-05T01:32:50.922 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 msgr2=0x7f6ec40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 0x7f6ec40a5050 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ed41980a0 tx=0x7f6ec0005d90 comp rx=0 tx=0).stop 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 msgr2=0x7f6ed4196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800d950 tx=0x7f6ec800de10 comp rx=0 tx=0).stop 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 shutdown_connections 2024-09-05T01:32:50.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f6ec40a2b90 0x7f6ec40a5050 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f6ed41025b0 0x7f6ed419e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6ed4101bf0 0x7f6ed41970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 --2- 172.21.15.129:0/1862064457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed4100a00 0x7f6ed4196b90 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 >> 172.21.15.129:0/1862064457 conn(0x7f6ed40fc140 msgr2=0x7f6ed4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 shutdown_connections 2024-09-05T01:32:50.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:50.923+0000 7f6edabd6640 1 -- 172.21.15.129:0/1862064457 wait complete. 2024-09-05T01:32:50.935 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T01:32:51.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1381421567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 msgr2=0x7fa1c4107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1381421567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c4107330 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009900 tx=0x7fa1b802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:51.040 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1381421567 shutdown_connections 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1381421567 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4107870 0x7fa1c4109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1381421567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c4107330 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1381421567 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4102610 0x7fa1c4104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1381421567 >> 172.21.15.129:0/1381421567 conn(0x7fa1c40fc190 msgr2=0x7fa1c40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1381421567 shutdown_connections 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.037+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1381421567 wait complete. 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 Processor -- start 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 -- start start 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 0x7fa1c4101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4107870 0x7fa1c4100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.041 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410d670 con 0x7fa1c4104f40 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410d4f0 con 0x7fa1c4102610 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c8d03640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c410d7f0 con 0x7fa1c4107870 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51308/0 (socket says 172.21.15.129:51308) 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 -- 172.21.15.129:0/1426250112 learned_addr learned my addr 172.21.15.129:0/1426250112 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c2d9d640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4107870 0x7fa1c4100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c259c640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 0x7fa1c4101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 -- 172.21.15.129:0/1426250112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4107870 msgr2=0x7fa1c4100b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4107870 0x7fa1c4100b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 -- 172.21.15.129:0/1426250112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 msgr2=0x7fa1c4101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 0x7fa1c4101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c1d9b640 1 -- 172.21.15.129:0/1426250112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c4101390 con 0x7fa1c4104f40 2024-09-05T01:32:51.042 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.038+0000 7fa1c259c640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 0x7fa1c4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1c1d9b640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80314e0 tx=0x7fa1b80042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b803f030 con 0x7fa1c4104f40 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1b8003cb0 con 0x7fa1c4104f40 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8035070 con 0x7fa1c4104f40 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c41a1f20 con 0x7fa1c4104f40 2024-09-05T01:32:51.043 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.039+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c41a2380 con 0x7fa1c4104f40 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.040+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c40fdc80 con 0x7fa1c4104f40 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.041+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa1b8042030 con 0x7fa1c4104f40 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.041+0000 7fa1b37fe640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 0x7fa1ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.042+0000 7fa1c259c640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 0x7fa1ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.042+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fa1b80ef9e0 con 0x7fa1c4104f40 2024-09-05T01:32:51.044 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.042+0000 7fa1c259c640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 0x7fa1ac0a5050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40058c0 tx=0x7fa1b4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.046 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.046+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1b80bc530 con 0x7fa1c4104f40 2024-09-05T01:32:51.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.213+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa1c4062ef0 con 0x7fa1c4104f40 2024-09-05T01:32:51.214 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.213+0000 7fa1b37fe640 1 -- 172.21.15.129:0/1426250112 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fa1b80bbe30 con 0x7fa1c4104f40 2024-09-05T01:32:51.214 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 64 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 msgr2=0x7fa1ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 0x7fa1ac0a5050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa1b40058c0 tx=0x7fa1b4005850 comp rx=0 tx=0).stop 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 msgr2=0x7fa1c41005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa1b80314e0 tx=0x7fa1b80042e0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 shutdown_connections 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa1ac0a2b90 0x7fa1ac0a5050 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa1c4107870 0x7fa1c4100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.217 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1c4104f40 0x7fa1c41005c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 --2- 172.21.15.129:0/1426250112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1c4102610 0x7fa1c4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 >> 172.21.15.129:0/1426250112 conn(0x7fa1c40fc190 msgr2=0x7fa1c4103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:51.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 shutdown_connections 2024-09-05T01:32:51.218 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.216+0000 7fa1c8d03640 1 -- 172.21.15.129:0/1426250112 wait complete. 2024-09-05T01:32:51.227 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3825281944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:32:51.227 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1756576076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.227 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:51 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1862064457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.228 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.328+0000 7f834cb18640 1 -- 172.21.15.129:0/2366294494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 msgr2=0x7f83481049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.328+0000 7f834cb18640 1 --2- 172.21.15.129:0/2366294494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f83481049f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8338009900 tx=0x7f833802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- 172.21.15.129:0/2366294494 shutdown_connections 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- 172.21.15.129:0/2366294494 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 0x7f8348109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- 172.21.15.129:0/2366294494 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 0x7f8348107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- 172.21.15.129:0/2366294494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f83481049f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- 172.21.15.129:0/2366294494 >> 172.21.15.129:0/2366294494 conn(0x7f83480fc140 msgr2=0x7f83480fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- 172.21.15.129:0/2366294494 shutdown_connections 2024-09-05T01:32:51.330 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- 172.21.15.129:0/2366294494 wait complete. 2024-09-05T01:32:51.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 Processor -- start 2024-09-05T01:32:51.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- start start 2024-09-05T01:32:51.332 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 0x7f8348198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 0x7f834819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f834810d670 con 0x7f8348102600 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f834810d4f0 con 0x7f8348104f30 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.329+0000 7f834cb18640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f834810d7f0 con 0x7f8348107860 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51330/0 (socket says 172.21.15.129:51330) 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 -- 172.21.15.129:0/1594799049 learned_addr learned my addr 172.21.15.129:0/1594799049 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f8346d9d640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 0x7f834819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f8345d9b640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 0x7f8348198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 -- 172.21.15.129:0/1594799049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 msgr2=0x7f834819f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 0x7f834819f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.333 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 -- 172.21.15.129:0/1594799049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 msgr2=0x7f8348198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 0x7f8348198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 -- 172.21.15.129:0/1594799049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834819fa40 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834659c640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8338004770 tx=0x7f83380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833803f030 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8338003cb0 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834819fcd0 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338035070 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.330+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83481a01c0 con 0x7f8348102600 2024-09-05T01:32:51.334 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.331+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83480fdc30 con 0x7f8348102600 2024-09-05T01:32:51.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.332+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f8338042030 con 0x7f8348102600 2024-09-05T01:32:51.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.333+0000 7f83377fe640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 0x7f83300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.333+0000 7f8345d9b640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 0x7f83300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.333+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f83380efff0 con 0x7f8348102600 2024-09-05T01:32:51.335 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.333+0000 7f8345d9b640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 0x7f83300a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f833c005fd0 tx=0x7f833c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.336 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.336+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83380bcb40 con 0x7f8348102600 2024-09-05T01:32:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3825281944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:32:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1756576076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:51 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1862064457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3825281944' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:32:51.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1756576076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:51 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1862064457' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T01:32:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.501+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f834802d7c0 con 0x7f8348102600 2024-09-05T01:32:51.502 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.502+0000 7f83377fe640 1 -- 172.21.15.129:0/1594799049 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f83380bc440 con 0x7f8348102600 2024-09-05T01:32:51.502 INFO:teuthology.orchestra.run.smithi129.stdout:pg_num: 32 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.503+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 msgr2=0x7f83300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.503+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 0x7f83300a5050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f833c005fd0 tx=0x7f833c005850 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.503+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 msgr2=0x7f8348197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8338004770 tx=0x7f83380042f0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 shutdown_connections 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f83300a2b90 0x7f83300a5050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f8348107860 0x7f834819f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8348104f30 0x7f8348198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 --2- 172.21.15.129:0/1594799049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8348102600 0x7f8348197e20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 >> 172.21.15.129:0/1594799049 conn(0x7f83480fc140 msgr2=0x7f8348102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 shutdown_connections 2024-09-05T01:32:51.505 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.504+0000 7f834cb18640 1 -- 172.21.15.129:0/1594799049 wait complete. 2024-09-05T01:32:51.516 DEBUG:teuthology.orchestra.run.smithi129:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-05T01:32:51.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 -- 172.21.15.129:0/2960626879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78100a30 msgr2=0x7efc78100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.616 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 --2- 172.21.15.129:0/2960626879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78100a30 0x7efc78100e30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7efc68009900 tx=0x7efc6802a410 comp rx=0 tx=0).stop 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 -- 172.21.15.129:0/2960626879 shutdown_connections 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 --2- 172.21.15.129:0/2960626879 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc781025e0 0x7efc78108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 --2- 172.21.15.129:0/2960626879 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78101c20 0x7efc781020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 --2- 172.21.15.129:0/2960626879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78100a30 0x7efc78100e30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 -- 172.21.15.129:0/2960626879 >> 172.21.15.129:0/2960626879 conn(0x7efc780fc1b0 msgr2=0x7efc780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 -- 172.21.15.129:0/2960626879 shutdown_connections 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.615+0000 7efc7e71f640 1 -- 172.21.15.129:0/2960626879 wait complete. 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 Processor -- start 2024-09-05T01:32:51.617 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 -- start start 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78100a30 0x7efc7806af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc781025e0 0x7efc7819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efc7810c540 con 0x7efc78101c20 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efc7810c3c0 con 0x7efc78100a30 2024-09-05T01:32:51.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc7e71f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7efc7810c6c0 con 0x7efc781025e0 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:51356/0 (socket says 172.21.15.129:51356) 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 -- 172.21.15.129:0/3305524271 learned_addr learned my addr 172.21.15.129:0/3305524271 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 -- 172.21.15.129:0/3305524271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc781025e0 msgr2=0x7efc7819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc781025e0 0x7efc7819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc77fff640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78100a30 0x7efc7806af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 -- 172.21.15.129:0/3305524271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78100a30 msgr2=0x7efc7806af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78100a30 0x7efc7806af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 -- 172.21.15.129:0/3305524271 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc7819fad0 con 0x7efc78101c20 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.616+0000 7efc777fe640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c890 tx=0x7efc6c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.617+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c004220 con 0x7efc78101c20 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.617+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc7819fd60 con 0x7efc78101c20 2024-09-05T01:32:51.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.617+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc6c0043c0 con 0x7efc78101c20 2024-09-05T01:32:51.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.617+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c004a00 con 0x7efc78101c20 2024-09-05T01:32:51.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.617+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc781a0250 con 0x7efc78101c20 2024-09-05T01:32:51.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.618+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc78107fb0 con 0x7efc78101c20 2024-09-05T01:32:51.621 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.620+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7efc6c004ba0 con 0x7efc78101c20 2024-09-05T01:32:51.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.620+0000 7efc757fa640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 0x7efc600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:32:51.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.620+0000 7efc77fff640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 0x7efc600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:32:51.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.620+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7efc6c0cf5f0 con 0x7efc78101c20 2024-09-05T01:32:51.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.621+0000 7efc77fff640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 0x7efc600a5050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efc68006010 tx=0x7efc68000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:32:51.624 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.623+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc6c014030 con 0x7efc78101c20 2024-09-05T01:32:51.837 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.836+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7efc7802da10 con 0x7efc78101c20 2024-09-05T01:32:51.844 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.843+0000 7efc757fa640 1 -- 172.21.15.129:0/3305524271 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740376 (secure 0 0 0) 0x7efc6c09c140 con 0x7efc78101c20 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "epoch": 20, 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "active_gid": 14150, 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "active_name": "x", 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "active_addrs": { 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-05T01:32:51.855 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6800", 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 655841650 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v1", 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:6801", 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 655841650 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "active_addr": "172.21.15.129:6801/655841650", 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "active_change": "2024-09-05T01:06:17.664396+0000", 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "active_mgr_features": 4540701547738038271, 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "available": true, 2024-09-05T01:32:51.856 INFO:teuthology.orchestra.run.smithi129.stdout: "standbys": [ 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "gid": 24112, 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "z", 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_features": 4540701547738038271, 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:51.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:51.858 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.859 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.860 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.861 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.862 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.866 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.867 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.868 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.869 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.870 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.871 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.873 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:51.874 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:51.875 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-09-05T01:32:51.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-09-05T01:32:51.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-09-05T01:32:51.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-09-05T01:32:51.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:51.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.881 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.882 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.883 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-09-05T01:32:51.887 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.888 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:51.889 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:51.890 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:51.891 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:51.892 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.893 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.894 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T01:32:51.896 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T01:32:51.897 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T01:32:51.898 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.899 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.900 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.902 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.903 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-09-05T01:32:51.904 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-09-05T01:32:51.905 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-09-05T01:32:51.906 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-09-05T01:32:51.907 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.908 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.911 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.912 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.913 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.914 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.915 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-09-05T01:32:51.917 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-09-05T01:32:51.918 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.919 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.920 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.921 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.922 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-09-05T01:32:51.923 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.924 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.925 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.926 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.929 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.930 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:51.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.932 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.933 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.934 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.935 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.936 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.938 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.940 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.941 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.942 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.943 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.944 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.945 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.946 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.947 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.948 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.949 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.951 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.952 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.953 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.954 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.955 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.956 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.959 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.960 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.961 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.962 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:51.963 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.964 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.965 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.966 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.967 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.968 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.970 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.971 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.972 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T01:32:51.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T01:32:51.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T01:32:51.975 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.976 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.978 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.980 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.981 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.982 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.983 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.984 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.985 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.986 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.987 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.988 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.989 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.990 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.991 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.992 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.993 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.994 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.995 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.996 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:51.998 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:51.999 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.001 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.002 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T01:32:52.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T01:32:52.005 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T01:32:52.006 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T01:32:52.007 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T01:32:52.008 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.009 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.010 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.011 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.012 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.016 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.017 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.018 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.020 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.021 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.022 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.023 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:52.024 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-09-05T01:32:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-09-05T01:32:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.039 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.043 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T01:32:52.044 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.046 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.047 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.048 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.049 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.051 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.052 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.053 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-09-05T01:32:52.054 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.055 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.056 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.057 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.058 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.059 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.060 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.061 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.062 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.063 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.064 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.065 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.066 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-09-05T01:32:52.067 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.069 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.070 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.071 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.072 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.073 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.074 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.076 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.077 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.079 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-09-05T01:32:52.081 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T01:32:52.082 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.083 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.084 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.085 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.087 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.088 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-09-05T01:32:52.089 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.090 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.091 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.093 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.094 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.095 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-09-05T01:32:52.096 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.097 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.098 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.099 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.100 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.101 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.102 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.103 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.104 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.106 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.107 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.108 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.110 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.111 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.112 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.113 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.114 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.117 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.118 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.119 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.120 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.121 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.122 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.123 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.124 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.126 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.127 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.128 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.129 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-09-05T01:32:52.131 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.132 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.136 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.137 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.138 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.139 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.140 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.141 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.142 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-09-05T01:32:52.144 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.145 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.147 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.148 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.149 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.150 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.151 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-09-05T01:32:52.152 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.153 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-09-05T01:32:52.154 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-09-05T01:32:52.155 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.156 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.157 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.158 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.159 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.160 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.161 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T01:32:52.162 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-09-05T01:32:52.163 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.164 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.165 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-09-05T01:32:52.166 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.167 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.168 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.169 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.170 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.171 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.172 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.173 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.174 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.175 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.176 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.177 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.178 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.179 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.180 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.181 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.182 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.183 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.184 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.185 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.186 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.187 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.188 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.189 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.190 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.191 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.192 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.193 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.194 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.195 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.196 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.197 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.198 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.199 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T01:32:52.200 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.201 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.202 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.203 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.204 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.205 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-09-05T01:32:52.206 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-09-05T01:32:52.207 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-09-05T01:32:52.208 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-09-05T01:32:52.209 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.210 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.211 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.212 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.213 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.214 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.215 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.216 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.217 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.221 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.222 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.223 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.224 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.225 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.226 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.227 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.228 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.229 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.230 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.231 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.232 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.233 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.234 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.235 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.236 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.237 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.239 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.240 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.241 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.242 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.243 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.244 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.245 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.246 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.247 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.248 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-09-05T01:32:52.249 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-09-05T01:32:52.250 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-09-05T01:32:52.251 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T01:32:52.252 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T01:32:52.253 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T01:32:52.254 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.255 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.256 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.257 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.258 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-09-05T01:32:52.259 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-09-05T01:32:52.260 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.261 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.262 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.263 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.264 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.265 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.266 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.267 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-09-05T01:32:52.268 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-09-05T01:32:52.269 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.270 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.271 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.272 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.273 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.275 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.276 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.277 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.278 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.279 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.280 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.281 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.282 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.283 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.284 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T01:32:52.285 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.286 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-09-05T01:32:52.287 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-09-05T01:32:52.288 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.289 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.290 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.291 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.292 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.293 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.294 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.295 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.296 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.297 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.298 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "gid": 24128, 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "y", 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "mgr_features": 4540701547738038271, 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-09-05T01:32:52.299 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T01:32:52.300 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.301 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.302 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.303 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.304 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.305 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.306 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.307 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.308 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.309 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-09-05T01:32:52.310 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.311 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.312 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.313 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-09-05T01:32:52.314 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T01:32:52.315 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T01:32:52.316 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.317 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.318 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.319 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-09-05T01:32:52.320 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-09-05T01:32:52.321 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.322 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.323 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.324 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.325 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.326 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.327 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-09-05T01:32:52.328 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:52.329 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.330 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.331 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.332 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.333 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.334 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.335 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.336 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.337 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T01:32:52.338 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.339 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.340 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.341 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.342 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.343 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.344 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-09-05T01:32:52.345 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-09-05T01:32:52.346 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-09-05T01:32:52.347 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.348 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.349 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.350 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.351 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-09-05T01:32:52.352 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.353 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.354 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.355 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T01:32:52.356 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.357 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.358 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.359 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.360 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.361 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.362 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-09-05T01:32:52.363 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-09-05T01:32:52.364 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.365 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.366 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.367 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.368 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.369 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.370 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.371 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-09-05T01:32:52.372 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.373 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.374 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.375 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-09-05T01:32:52.376 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.377 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.378 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.379 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.380 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.381 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T01:32:52.382 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-09-05T01:32:52.383 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.384 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.385 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.386 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.387 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.388 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.389 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.390 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.391 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.392 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.393 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.394 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.395 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-09-05T01:32:52.396 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-09-05T01:32:52.397 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.398 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.399 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.400 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.401 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.402 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.403 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.404 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.405 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.406 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.407 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.408 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.409 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.410 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.411 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T01:32:52.412 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T01:32:52.413 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.414 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T01:32:52.415 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T01:32:52.416 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.417 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.420 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.420 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.420 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T01:32:52.420 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.421 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.422 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.423 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.424 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.425 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.426 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.427 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.428 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.429 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.430 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.431 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.432 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.433 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.434 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.435 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.436 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.437 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.438 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.439 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.440 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.441 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.442 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.443 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.444 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.445 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.446 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.447 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.448 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.449 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T01:32:52.450 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T01:32:52.451 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T01:32:52.452 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.453 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.454 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.455 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.456 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.457 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.458 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.459 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.460 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.461 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.462 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.463 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.464 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.465 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.466 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.467 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.468 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.469 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.470 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.471 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.472 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.473 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.474 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.475 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.476 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.477 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.478 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.479 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.480 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.481 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.482 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.483 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.484 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-09-05T01:32:52.485 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-09-05T01:32:52.486 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-09-05T01:32:52.487 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-09-05T01:32:52.488 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-09-05T01:32:52.489 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.490 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.491 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.492 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.493 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.494 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.495 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.496 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.497 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.498 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.499 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.500 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.501 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.502 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.503 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.504 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.505 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.506 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.507 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.508 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.509 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.510 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.511 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.512 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.513 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-09-05T01:32:52.514 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-09-05T01:32:52.515 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.516 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.517 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.518 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.520 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.521 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.522 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.523 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.524 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.525 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.526 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.527 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.528 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.529 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.530 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.531 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.532 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.534 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.535 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.536 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.538 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.539 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.540 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.541 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.542 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T01:32:52.543 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.544 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.545 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.546 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.547 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.548 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.549 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.550 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.551 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.552 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.553 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.554 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.555 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.556 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.557 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.558 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.559 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.560 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.561 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.562 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.563 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.564 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.565 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.566 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.567 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.568 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.569 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.570 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.571 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-09-05T01:32:52.572 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.573 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.574 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.575 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.576 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.577 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.578 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.579 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.580 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.581 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.582 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.583 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.584 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.585 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.586 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.587 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.588 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.589 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.590 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.591 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.592 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.593 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.594 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.595 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.596 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.597 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.598 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.599 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.600 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.601 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.602 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.603 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.604 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.605 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.606 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.607 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.608 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.609 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.610 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.611 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.612 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.613 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.614 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.615 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.616 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.617 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.618 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.619 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.620 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.621 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.622 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.623 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.623 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.623 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.624 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-09-05T01:32:52.624 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-09-05T01:32:52.624 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T01:32:52.625 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.626 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.627 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.628 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.629 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.630 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.631 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.632 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.633 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.634 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.635 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.636 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.637 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.638 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.639 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.640 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.641 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.642 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.643 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.644 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.645 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.646 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.647 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.648 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-09-05T01:32:52.649 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.650 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.651 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.652 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.653 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.654 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.655 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-09-05T01:32:52.656 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.657 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.658 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.659 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.660 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.661 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.662 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.663 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.664 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.665 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.666 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.667 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.668 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.669 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.670 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.671 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.672 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.673 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.674 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.675 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.676 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.677 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.678 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.679 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.680 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.681 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.682 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.683 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.684 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.685 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-09-05T01:32:52.686 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.687 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.688 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.689 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.690 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.691 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.692 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.693 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.694 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.695 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.696 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.697 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.698 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.699 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.700 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.701 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.702 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.703 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.704 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.705 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.706 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.707 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.708 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.709 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.710 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.711 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-09-05T01:32:52.712 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-09-05T01:32:52.713 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-09-05T01:32:52.714 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.715 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.716 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.717 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.718 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.719 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.720 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-09-05T01:32:52.721 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.722 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.723 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.724 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.725 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.726 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.727 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.728 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T01:32:52.729 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T01:32:52.730 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.731 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.732 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.733 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.734 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.735 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.736 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.737 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.738 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-09-05T01:32:52.739 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-09-05T01:32:52.740 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.741 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.742 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "modules": [ 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "dashboard", 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "iostat", 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "nfs", 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "restful" 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "available_modules": [ 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "alerts", 2024-09-05T01:32:52.743 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.744 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.745 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.746 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.747 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.748 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.749 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_destination": { 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_destination", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.750 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email address to send alerts to", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_from_name": { 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_from_name", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.751 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "Ceph", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Email From: name", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_host": { 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_host", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.752 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP server", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_password": { 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_password", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.753 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Password to authenticate with", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.754 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_port": { 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_port", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "465", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP port", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.755 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_sender": { 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_sender", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SMTP envelope sender", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.756 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_ssl": { 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_ssl", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.757 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "smtp_user": { 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "smtp_user", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "User to authenticate as", 2024-09-05T01:32:52.758 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "balancer", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "active": { 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "active", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.759 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_time": { 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_time", 2024-09-05T01:32:52.760 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0000", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "begin_weekday": { 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "begin_weekday", 2024-09-05T01:32:52.761 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_max_iterations": { 2024-09-05T01:32:52.762 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_max_iterations", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "25", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "250", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.763 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_metrics": { 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_metrics", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.764 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "crush_compat_step": { 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crush_compat_step", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.5", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0.001", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "0.999", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "aggressiveness of optimization", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.765 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "end_time": { 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_time", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2359", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T01:32:52.766 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "end_weekday": { 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "end_weekday", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "0", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "6", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T01:32:52.767 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.768 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.769 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.770 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.771 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.772 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "min_score": { 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_score", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T01:32:52.773 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "upmap", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.774 INFO:teuthology.orchestra.run.smithi129.stdout: "crush-compat", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "none", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "read", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap-read" 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Balancer mode", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_ids": { 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_ids", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.775 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.776 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_deviation": { 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_deviation", 2024-09-05T01:32:52.777 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "upmap_max_optimizations": { 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "upmap_max_optimizations", 2024-09-05T01:32:52.778 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.779 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_down_multiplier": { 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_down_multiplier", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.780 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_refresh_rate": { 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_refresh_rate", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "20", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.781 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "agent_starting_port": { 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "agent_starting_port", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4721", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.782 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_ptrace": { 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_ptrace", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.783 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_interval": { 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_interval", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.784 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "autotune_memory_target_ratio": { 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0.7", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.785 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm_log_destination": { 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cephadm_log_destination", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.786 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "file", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "file,syslog", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "syslog" 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "cgroups_split": { 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cgroups_split", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.787 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "config_checks_enabled": { 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_checks_enabled", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.788 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "config_dashboard": { 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_dashboard", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.789 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_alertmanager": { 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_alertmanager", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.790 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_base": { 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_base", 2024-09-05T01:32:52.791 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Container image name, without the tag", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_elasticsearch": { 2024-09-05T01:32:52.792 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_elasticsearch", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "elasticsearch container image", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.793 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_grafana": { 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_grafana", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.794 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_haproxy": { 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_haproxy", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "HAproxy container image", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.795 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_agent": { 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_agent", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger agent container image", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.796 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_collector": { 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_collector", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger collector container image", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.797 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_jaeger_query": { 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_jaeger_query", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Jaeger query container image", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.798 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_keepalived": { 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_keepalived", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Keepalived container image", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.799 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_loki": { 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_loki", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Loki container image", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.800 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_node_exporter": { 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_node_exporter", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.801 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_nvmeof": { 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_nvmeof", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Nvme-of container image", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.802 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_prometheus": { 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_prometheus", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Prometheus container image", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.803 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_promtail": { 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_promtail", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Promtail container image", 2024-09-05T01:32:52.804 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_samba": { 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_samba", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Samba/SMB container image", 2024-09-05T01:32:52.805 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "container_image_snmp_gateway": { 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_image_snmp_gateway", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "SNMP Gateway container image", 2024-09-05T01:32:52.806 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "container_init": { 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "container_init", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.807 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "daemon_cache_timeout": { 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "daemon_cache_timeout", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.808 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "default_cephadm_command_timeout": { 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "900", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.809 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "default_registry": { 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "default_registry", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "docker.io", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.810 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "device_cache_timeout": { 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_cache_timeout", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1800", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.811 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache device inventory", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "device_enhanced_scan": { 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_enhanced_scan", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.812 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "facts_cache_timeout": { 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "facts_cache_timeout", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.813 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "seconds to cache host facts data", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "grafana_dashboards_path": { 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "grafana_dashboards_path", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.814 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "host_check_interval": { 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "host_check_interval", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.815 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to perform a host check", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "hw_monitoring": { 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hw_monitoring", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.816 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "inventory_list_all": { 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "inventory_list_all", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.817 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.818 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "log_refresh_metadata": { 2024-09-05T01:32:52.819 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_refresh_metadata", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.820 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.821 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.822 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.823 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.824 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.825 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "*", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "max_count_per_host": { 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_count_per_host", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.826 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of daemons per service per host", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "max_osd_draining_count": { 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_osd_draining_count", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.827 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "migration_current": { 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "migration_current", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.828 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "internal - do not modify", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "mode": { 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mode", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.829 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "root", 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm-package", 2024-09-05T01:32:52.830 INFO:teuthology.orchestra.run.smithi129.stdout: "root" 2024-09-05T01:32:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T01:32:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.832 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "oob_default_addr": { 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "oob_default_addr", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "169.254.1.1", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T01:32:52.833 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "prometheus_alerts_path": { 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus_alerts_path", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.834 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_insecure": { 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_insecure", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.835 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_password": { 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_password", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.836 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_url": { 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_url", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.837 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "registry_username": { 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "registry_username", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.838 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "secure_monitoring_stack": { 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "secure_monitoring_stack", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.839 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "service_discovery_port": { 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "service_discovery_port", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.840 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8765", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "cephadm service discovery port", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "ssh_config_file": { 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssh_config_file", 2024-09-05T01:32:52.841 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "use_agent": { 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_agent", 2024-09-05T01:32:52.842 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "use_repo_digest": { 2024-09-05T01:32:52.843 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "use_repo_digest", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.844 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_failed_host_check": { 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_failed_host_check", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.845 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_daemons": { 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_daemons", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.846 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_on_stray_hosts": { 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_on_stray_hosts", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T01:32:52.847 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crash", 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.848 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.849 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.850 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.851 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.852 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.853 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "retain_interval": { 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retain_interval", 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.854 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "31536000", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_recent_interval": { 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_recent_interval", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.855 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1209600", 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.856 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dashboard", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.857 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.858 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.859 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_ENABLED": { 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.860 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.861 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.862 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T01:32:52.863 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T01:32:52.864 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.865 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.866 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.867 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.868 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.869 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.870 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.871 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.872 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.873 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_URL": { 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_URL", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.874 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.875 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.876 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.877 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.878 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.879 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.880 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.881 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T01:32:52.882 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T01:32:52.883 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.884 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.885 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.886 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.887 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.888 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.889 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.890 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.891 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.892 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.893 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "45", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.894 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.895 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.896 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "admin", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.897 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.898 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.899 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.900 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T01:32:52.901 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "cross_origin_url": { 2024-09-05T01:32:52.902 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cross_origin_url", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "crt_file": { 2024-09-05T01:32:52.903 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "crt_file", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "debug": { 2024-09-05T01:32:52.904 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "debug", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Enable/disable debug options", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "jwt_token_ttl": { 2024-09-05T01:32:52.905 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "jwt_token_ttl", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "28800", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:52.906 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.907 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.908 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.909 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.910 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.911 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.912 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "motd": { 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "motd", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "The message of the day", 2024-09-05T01:32:52.913 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect_resolve_ip_addr": { 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.914 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.915 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8080", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.916 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.917 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl_server_port": { 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl_server_port", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8443", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.918 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "redirect", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.919 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "redirect" 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:52.920 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "url_prefix": { 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url_prefix", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.921 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.922 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_monitoring": { 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_monitoring", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "monitor device health metrics", 2024-09-05T01:32:52.923 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.924 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.925 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.926 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.927 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.928 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "mark_out_threshold": { 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mark_out_threshold", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "2419200", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.929 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "pool_name": { 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pool_name", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "device_health_metrics", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.930 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "retention_period": { 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "retention_period", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15552000", 2024-09-05T01:32:52.931 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long to retain device health metrics", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_frequency": { 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_frequency", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.932 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "self_heal": { 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "self_heal", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.933 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.934 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "warn_threshold": { 2024-09-05T01:32:52.935 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "warn_threshold", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7257600", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.936 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "diskprediction_local", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.937 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.938 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.939 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.940 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.941 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "predict_interval": { 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predict_interval", 2024-09-05T01:32:52.942 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "86400", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.943 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "predictor_model": { 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "predictor_model", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "prophetstor", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.944 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "600", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.945 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "influx", 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": false, 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "influxdb python module not found", 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "batch_size": { 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "batch_size", 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.946 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5000", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "database": { 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "database", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.947 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "ceph", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "hostname": { 2024-09-05T01:32:52.948 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "hostname", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server hostname", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:52.949 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "30", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "5", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.950 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.951 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.952 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.953 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.954 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.955 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "password": { 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "password", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.956 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "password of InfluxDB server user", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "port": { 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "port", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.957 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "8086", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "InfluxDB server port", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "ssl": { 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ssl", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.958 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "false", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "threads": { 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threads", 2024-09-05T01:32:52.959 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "32", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.960 INFO:teuthology.orchestra.run.smithi129.stdout: "username": { 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "username", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "username of InfluxDB server user", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.961 INFO:teuthology.orchestra.run.smithi129.stdout: "verify_ssl": { 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "verify_ssl", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "true", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.962 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "insights", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.963 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.964 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.965 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.966 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.967 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "iostat", 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.968 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.969 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.970 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.971 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.972 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.973 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "k8sevents", 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_event_retention_days": { 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "ceph_event_retention_days", 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.974 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "7", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "config_check_secs": { 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "config_check_secs", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.975 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "10", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.976 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.977 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.978 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.979 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.980 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.981 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "localpool", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "failure_domain": { 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "failure_domain", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "host", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.982 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "failure domain for any created local pool", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.983 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.984 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.985 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.986 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.987 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "min_size": { 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "min_size", 2024-09-05T01:32:52.988 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default min_size for any created local pool", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "num_rep": { 2024-09-05T01:32:52.989 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "num_rep", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default replica count for any created local pool", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_num": { 2024-09-05T01:32:52.990 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_num", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "128", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "prefix": { 2024-09-05T01:32:52.991 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prefix", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "name prefix for any created local pool", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.992 INFO:teuthology.orchestra.run.smithi129.stdout: "subtree": { 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "subtree", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "rack", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.993 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mds_autoscaler", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.994 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.996 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:52.997 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:52.998 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:52.999 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirroring", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.000 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.001 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.002 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.003 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.004 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "nfs", 2024-09-05T01:32:53.005 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.006 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.007 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.008 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.009 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.010 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.011 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "fail_fs": { 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "fail_fs", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T01:32:53.012 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.013 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.014 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.015 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.016 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.017 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator": { 2024-09-05T01:32:53.018 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "orchestrator", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "cephadm", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "rook", 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "test_orchestrator" 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.019 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Orchestrator backend", 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_perf_query", 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.020 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.021 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.022 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.023 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.024 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.025 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "osd_support", 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.026 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.027 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.028 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.029 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.030 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.031 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "pg_autoscaler", 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.032 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.033 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.034 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.035 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.036 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.037 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.038 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "threshold": { 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "threshold", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "3.0", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1.0", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.039 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "scaling threshold", 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "progress", 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.040 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_pg_recovery_event": { 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_pg_recovery_event", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.041 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.042 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.043 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.044 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.045 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.046 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.047 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "max_completed_events": { 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_completed_events", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "50", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "number of past completed events to remember", 2024-09-05T01:32:53.048 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "sleep_interval": { 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "sleep_interval", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "5", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "how long the module is going to sleep", 2024-09-05T01:32:53.049 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "prometheus", 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "cache": { 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "cache", 2024-09-05T01:32:53.050 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "exclude_perf_counters": { 2024-09-05T01:32:53.051 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "exclude_perf_counters", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.052 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.053 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.054 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.055 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.056 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.057 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools": { 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.058 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.059 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "300", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "scrape_interval": { 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "scrape_interval", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:53.060 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15.0", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.061 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "::", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:53.062 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "9283", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "stale_cache_strategy": { 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stale_cache_strategy", 2024-09-05T01:32:53.063 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "log", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_behaviour": { 2024-09-05T01:32:53.064 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_behaviour", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "default", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "default", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "error" 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.065 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "standby_error_status_code": { 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "standby_error_status_code", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "500", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "400", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "599", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.066 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.067 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.068 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.069 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.070 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.071 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.072 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_snap_create": { 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_snap_create", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.073 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "mirror_snapshot_schedule": { 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.074 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "trash_purge_schedule": { 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "trash_purge_schedule", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.075 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "restful", 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "enable_auth": { 2024-09-05T01:32:53.076 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enable_auth", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "key_file": { 2024-09-05T01:32:53.077 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "key_file", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.078 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.079 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.080 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.081 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.082 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.083 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "server_addr": { 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_addr", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.084 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "server_port": { 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "server_port", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.085 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rgw", 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.086 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.087 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.088 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.089 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.090 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.091 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rook", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.092 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.093 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.094 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.095 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.096 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "storage_class": { 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "storage_class", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "local", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T01:32:53.097 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "selftest", 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.098 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.099 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.100 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.101 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.102 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.103 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "roption1": { 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption1", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.104 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "roption2": { 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "roption2", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "xyz", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.105 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption1": { 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption1", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.106 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption2": { 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption2", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.107 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption3": { 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption3", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "float", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.108 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption4": { 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption4", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.109 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption5": { 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption5", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.110 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption6": { 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption6", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.111 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "rwoption7": { 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rwoption7", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.112 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "1", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "42", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "testkey": { 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testkey", 2024-09-05T01:32:53.113 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "testlkey": { 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testlkey", 2024-09-05T01:32:53.114 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "testnewline": { 2024-09-05T01:32:53.115 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "testnewline", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.116 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snap_schedule", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "allow_m_granularity": { 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "allow_m_granularity", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.117 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "dump_on_update": { 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "dump_on_update", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.118 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "dump database to debug log on update", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.119 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.120 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.121 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.122 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.123 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.124 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "stats", 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.125 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.126 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.127 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.128 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.129 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.130 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "status", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.131 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.132 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.133 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.134 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.135 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telegraf", 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "address": { 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "address", 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.136 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:53.137 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "15", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.138 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.139 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.140 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.141 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.142 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.143 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "telemetry", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_basic": { 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_basic", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.144 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_crash": { 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_crash", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.145 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_device": { 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_device", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.146 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_ident": { 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_ident", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.147 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "channel_perf": { 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "channel_perf", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.148 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "contact": { 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "contact", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.149 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "description": { 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "description", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.150 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "device_url": { 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "device_url", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.151 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "enabled": { 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "enabled", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.152 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.153 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "24", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "8", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "last_opt_revision": { 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "last_opt_revision", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.154 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "1", 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard": { 2024-09-05T01:32:53.155 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.156 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "leaderboard_description": { 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "leaderboard_description", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.157 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.158 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.159 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.160 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.161 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.162 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "organization": { 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "organization", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.163 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "proxy": { 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "proxy", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.164 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "url": { 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "url", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.165 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "test_orchestrator", 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.166 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.167 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.168 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.169 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.170 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.171 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.172 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.173 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.174 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.175 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.176 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "max_concurrent_clones": { 2024-09-05T01:32:53.177 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "max_concurrent_clones", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "4", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.178 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "periodic_async_work": { 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "periodic_async_work", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Periodically check for async work", 2024-09-05T01:32:53.179 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_delay": { 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_delay", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "0", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.180 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "snapshot_clone_no_wait": { 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "True", 2024-09-05T01:32:53.181 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "can_run": true, 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "error_string": "", 2024-09-05T01:32:53.182 INFO:teuthology.orchestra.run.smithi129.stdout: "module_options": { 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "discovery_interval": { 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "discovery_interval", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "uint", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "100", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.183 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "identifier": { 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "identifier", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.184 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "interval": { 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "interval", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "secs", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "60", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.185 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "log_level": { 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_level", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.186 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster": { 2024-09-05T01:32:53.187 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_cluster_level": { 2024-09-05T01:32:53.188 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_cluster_level", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "info", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [ 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "critical", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "debug", 2024-09-05T01:32:53.189 INFO:teuthology.orchestra.run.smithi129.stdout: "error", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "info", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "warning" 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "log_to_file": { 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "log_to_file", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "bool", 2024-09-05T01:32:53.190 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 1, 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "False", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.191 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_host": { 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_host", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.192 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_port": { 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_port", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "int", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "10051", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.193 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "zabbix_sender": { 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "zabbix_sender", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "str", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "level": "advanced", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "flags": 0, 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "min": "", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "max": "", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "enum_allowed": [], 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "desc": "", 2024-09-05T01:32:53.194 INFO:teuthology.orchestra.run.smithi129.stdout: "long_desc": "", 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "tags": [], 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "see_also": [] 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "services": { 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "dashboard": "https://172.21.15.129:8443/" 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "always_on_modules": { 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "octopus": [ 2024-09-05T01:32:53.195 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "pacific": [ 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-09-05T01:32:53.196 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "quincy": [ 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-09-05T01:32:53.197 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "reef": [ 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-09-05T01:32:53.198 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: ], 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "squid": [ 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "balancer", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "crash", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "devicehealth", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "orchestrator", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "pg_autoscaler", 2024-09-05T01:32:53.199 INFO:teuthology.orchestra.run.smithi129.stdout: "progress", 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "rbd_support", 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "status", 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "telemetry", 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "volumes" 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "last_failure_osd_epoch": 3, 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "active_clients": [ 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "devicehealth", 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.200 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 3866894450 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "libcephsqlite", 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2101621157 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.201 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "rbd_support", 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2624723288 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "name": "volumes", 2024-09-05T01:32:53.202 INFO:teuthology.orchestra.run.smithi129.stdout: "addrvec": [ 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: { 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: "type": "v2", 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: "addr": "172.21.15.129:0", 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: "nonce": 2651423209 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-05T01:32:53.203 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:53.204 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:52 smithi194 ceph-mon[34941]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.4 MiB/s wr, 589 op/s 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:52 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1426250112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:52 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1594799049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:52 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3305524271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:52 smithi183 ceph-mon[34589]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.4 MiB/s wr, 589 op/s 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:52 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1426250112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:52 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1594799049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.205 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:52 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3305524271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 msgr2=0x7efc600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 0x7efc600a5050 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7efc68006010 tx=0x7efc68000bc0 comp rx=0 tx=0).stop 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 msgr2=0x7efc7819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c890 tx=0x7efc6c00cd50 comp rx=0 tx=0).stop 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 shutdown_connections 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7efc600a2b90 0x7efc600a5050 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:53.206 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7efc781025e0 0x7efc7819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:53.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efc78101c20 0x7efc7819cba0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:53.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 --2- 172.21.15.129:0/3305524271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efc78100a30 0x7efc7806af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:32:53.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 >> 172.21.15.129:0/3305524271 conn(0x7efc780fc1b0 msgr2=0x7efc78105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:32:53.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 shutdown_connections 2024-09-05T01:32:53.207 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:32:51.849+0000 7efc7e71f640 1 -- 172.21.15.129:0/3305524271 wait complete. 2024-09-05T01:32:53.276 DEBUG:tasks.check_counter:Getting stats from a 2024-09-05T01:32:53.277 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-05T01:32:53.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:52 smithi129 ceph-mon[30979]: pgmap v879: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 5.4 MiB/s wr, 589 op/s 2024-09-05T01:32:53.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:52 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1426250112' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.277 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:52 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1594799049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T01:32:53.278 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:52 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3305524271' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T01:32:53.536 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 4, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 4, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 3040, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 448, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-09-05T01:32:54.071 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.003706919, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000032023, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.002206094, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.003997443, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001332481 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 2, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 245, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 320, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 200560, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.022151338, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.017025474, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001629708, 2024-09-05T01:32:54.073 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 243, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.018568451, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000076413 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 200560 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.074 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 325, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 320, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 783968, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 197680, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.052347297, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.025450328, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.032209129, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 319, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.019003046, 2024-09-05T01:32:54.075 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000059570 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 783968, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 197680 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-05T01:32:54.076 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:32:54.077 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:32:54.078 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148761, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 31, 2024-09-05T01:32:54.079 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-05T01:32:54.080 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 327, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 738050, 2024-09-05T01:32:54.081 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 327, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 327, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 738050, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:54.082 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:54.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:54 smithi129 ceph-mon[30979]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-05T01:32:54.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:54 smithi183 ceph-mon[34589]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-05T01:32:54.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:54 smithi194 ceph-mon[34941]: pgmap v880: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-05T01:32:54.887 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-05T01:32:54.888 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-05T01:32:54.888 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-05T01:32:54.888 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-05T01:32:54.888 DEBUG:tasks.check_counter:Getting stats from d 2024-09-05T01:32:54.888 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-05T01:32:55.136 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-09-05T01:32:55.615 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.001807714, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000365889, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.616 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 6, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 251, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 3360, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 201712, 2024-09-05T01:32:55.617 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.023905624, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.017104680, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.003573676, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 250, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.020234902, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000080939 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:55.618 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 201712 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 330, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 325, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 785328, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 200880, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-09-05T01:32:55.619 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.059938990, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.026963407, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.039424705, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 324, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.023462731, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000072415 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:55.620 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 785328, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 200880 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:32:55.621 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:32:55.622 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-05T01:32:55.623 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148721, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 30, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:32:55.624 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-05T01:32:55.625 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 332, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 738700, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 332, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 332, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 738700, 2024-09-05T01:32:55.626 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:55.627 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:56.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:56 smithi129 ceph-mon[30979]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-05T01:32:56.349 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-05T01:32:56.349 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-05T01:32:56.349 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-05T01:32:56.349 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-05T01:32:56.349 DEBUG:tasks.check_counter:Getting stats from g 2024-09-05T01:32:56.349 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-05T01:32:56.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:56 smithi183 ceph-mon[34589]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-05T01:32:56.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:56 smithi194 ceph-mon[34941]: pgmap v881: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 2.3 MiB/s wr, 269 op/s 2024-09-05T01:32:56.610 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 4, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 4, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 3040, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 448, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-09-05T01:32:57.126 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.003070768, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000033558, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.002186453, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.004084604, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.001361534 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:57.127 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 326, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 321, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 784240, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 198320, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.055440671, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.024342820, 2024-09-05T01:32:57.128 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.037246147, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 320, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.029181824, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000091193 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 784240, 2024-09-05T01:32:57.129 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 198320 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 2, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 246, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 320, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 200736, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.021774860, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.016141687, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.003409265, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:57.130 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 244, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.018315231, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000075062 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 200736 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.131 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:32:57.132 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:32:57.133 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:32:57.134 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148704, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 29, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-05T01:32:57.135 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:57.136 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 328, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 738180, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 328, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 328, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 738180, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:57.137 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:57.138 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:57.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:57.138 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:57.138 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:57.841 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-05T01:32:57.841 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-05T01:32:57.841 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-05T01:32:57.841 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-05T01:32:57.841 DEBUG:tasks.check_counter:Getting stats from j 2024-09-05T01:32:57.841 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-05T01:32:58.093 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:32:58.387 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:32:58 smithi129 ceph-mon[30979]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 545 KiB/s wr, 77 op/s 2024-09-05T01:32:58.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:32:58 smithi183 ceph-mon[34589]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 545 KiB/s wr, 77 op/s 2024-09-05T01:32:58.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:32:58 smithi194 ceph-mon[34941]: pgmap v882: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 545 KiB/s wr, 77 op/s 2024-09-05T01:32:58.604 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-05T01:32:58.604 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:32:58.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-09-05T01:32:58.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-09-05T01:32:58.604 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.001187287, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000292403, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:58.605 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 335, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 328, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 788640, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 201968, 2024-09-05T01:32:58.606 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.064597375, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.025557198, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.045116174, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 328, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.028920304, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000088171 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:58.607 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 788640, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 201968 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 2, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 250, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 320, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 201440, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-05T01:32:58.608 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.020394668, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.015975454, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001692884, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 248, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.016560146, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000066774 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.609 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 201440 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:32:58.610 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:32:58.611 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-05T01:32:58.612 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148744, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 30, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-05T01:32:58.613 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-05T01:32:58.614 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 333, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 738830, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 333, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 333, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 738830, 2024-09-05T01:32:58.615 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-05T01:32:58.616 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-05T01:32:59.320 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-05T01:32:59.320 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-05T01:32:59.320 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-05T01:32:59.320 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-05T01:32:59.320 DEBUG:tasks.check_counter:Getting stats from b 2024-09-05T01:32:59.320 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-05T01:32:59.729 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:33:00.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:00 smithi183 ceph-mon[34589]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-09-05T01:33:00.411 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-05T01:33:00.411 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6497, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 9699, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1708176, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 22901855, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 215, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.652868699, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.274232669, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.276137645, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.085646284, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9700, 2024-09-05T01:33:00.412 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.008722831, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000103992 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9493, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002211406, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000232 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 41424, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 5136 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6875, 2024-09-05T01:33:00.413 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 10399, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1827047, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 21831587, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 31, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.540783158, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.207779148, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.218571950, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.105128703, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 10401, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4.061078358, 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000390450 2024-09-05T01:33:00.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9273, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002713743, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000292 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 519648 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 45857, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 33574, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 16030127, 2024-09-05T01:33:00.415 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 24948341, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 17, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 8, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 2.462551842, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.927895026, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 1.905806913, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.140430458, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 33567, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.385670024, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000071071 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 69323, 2024-09-05T01:33:00.416 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.010685142, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000154 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 924432, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 236208 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:00.417 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7811, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8.240388622, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001054972 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 992, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.040734613, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000041063 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-05T01:33:00.418 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 9451, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 9446, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9446, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 97.316265212, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.010302378 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 5, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 1362, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 3, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-05T01:33:00.419 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 1098, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 6, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 1092, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 1051, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 2, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 1038, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 1038, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 6, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 16174, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 13558, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 1, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-05T01:33:00.420 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 25, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 2, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 2, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 1013, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 1450841979, 2024-09-05T01:33:00.421 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 3793, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 4423, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 1, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:00.422 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 31580, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 10708, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 68, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 2703 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 84, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 138, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:00.423 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 409, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 251, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 158, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 17, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 17695, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 2, 2024-09-05T01:33:00.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 3, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 18055, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 18045, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 18045, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 10, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 330, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 91, 2024-09-05T01:33:00.425 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 90, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 90, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 1, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 1, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 1, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 45461174, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 45488509, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3518, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 22.800672324, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.006481146 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.426 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 1059, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 1312, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 253, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 68, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 73, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 5, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 1098, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1672, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 574, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 1038, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 1306, 2024-09-05T01:33:00.427 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 268, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 78904, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 231712 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 10579, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 1, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 9451, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 313, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 2, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1262, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 89.709980615, 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.071085563 2024-09-05T01:33:00.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 466, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.546497294, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001172740 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-05T01:33:00.429 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 158, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.293949620, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001860440 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5168, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.620479349, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000700557 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.430 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.432 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 37, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.044898853, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001213482 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.433 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1669, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.037806806, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000621813 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 93, 2024-09-05T01:33:00.434 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.186781844, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002008406 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 391, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.580433621, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004042029 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.435 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.008250675, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004125337 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.436 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.002974573, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000991524 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:00.437 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021918181, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002435353 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 188, 2024-09-05T01:33:00.439 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.262293781, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001395179 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 4, 2024-09-05T01:33:00.440 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 3, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 1994, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 1994, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1052, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:00.441 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:00.442 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:00.443 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 286327, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 93, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:00.444 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 174, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 3590161, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 30678, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.445 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 9569, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 45493564, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 32, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 9569, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9569, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 373.767100285, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.039060204 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9569, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 16557 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 9569, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 127, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 9442, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 1293, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 2460, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 127, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 3645, 2024-09-05T01:33:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 385, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 3, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 1065, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 3781, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 2, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-05T01:33:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 93, 2024-09-05T01:33:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 13, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 3474, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 322 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 12467 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 29, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 8, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 535, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 251, 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 59225, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 14201809, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 59225, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 59225, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 14201809, 2024-09-05T01:33:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 9569, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 45520907, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9569, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 45520907, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 9569, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 9569, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 9569, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 9569, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 251, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 27343, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 251, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 127, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 27343, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55f18ed90960": { 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 18055, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 41294205, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 18055, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3518, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 41294205, 2024-09-05T01:33:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:00.459 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-05T01:33:00.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:00 smithi129 ceph-mon[30979]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-09-05T01:33:00.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:00 smithi194 ceph-mon[34941]: pgmap v883: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 22 op/s 2024-09-05T01:33:01.464 INFO:tasks.check_counter:Daemon mds.b mds.exported=2 2024-09-05T01:33:01.464 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-05T01:33:01.464 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=84 2024-09-05T01:33:01.464 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-05T01:33:01.464 DEBUG:tasks.check_counter:Getting stats from e 2024-09-05T01:33:01.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-05T01:33:01.716 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1170, 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1170, 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 342773, 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 371300, 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 7, 2024-09-05T01:33:02.272 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.092983931, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.036295860, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.023598935, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.023878996, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1169, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.045289187, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000038741 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.273 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1410, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1395, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1104716, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 539715, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.125901775, 2024-09-05T01:33:02.274 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.043576090, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.053099177, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.031310451, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1394, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.048371160, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000034699 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 844672, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 203584 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1151, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 1398, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 386560, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 868852, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.130796005, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.045929171, 2024-09-05T01:33:02.276 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.031955615, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.039769852, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1396, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.062593208, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000044837 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T01:33:02.277 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 503152 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5586, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.141672509, 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000025362 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-05T01:33:02.279 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 11, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-09-05T01:33:02.280 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 11, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 11, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:02.281 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-09-05T01:33:02.282 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:02.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-05T01:33:02.284 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-09-05T01:33:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-05T01:33:02.286 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 22, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4228114, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 4227540, 2024-09-05T01:33:02.287 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 22 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 14, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 14, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 0, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 11, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 11, 2024-09-05T01:33:02.288 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 11, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 11, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 51016, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 215328 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-09-05T01:33:02.289 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:02.290 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.292 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-05T01:33:02.293 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-05T01:33:02.295 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.297 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.298 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-05T01:33:02.299 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.300 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 1, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 0, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-05T01:33:02.301 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2127, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:02.302 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:02.303 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:02.304 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 190145, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 52, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 174, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:02.305 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 53165, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 170, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 3373, 2024-09-05T01:33:02.306 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 3373, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3373, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.782962631, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001121542 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3373, 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3374 2024-09-05T01:33:02.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 3373, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 3373, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 2205, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 1166, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-05T01:33:02.308 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-05T01:33:02.309 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-05T01:33:02.310 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 93, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 7, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 6, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 1, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-09-05T01:33:02.311 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-05T01:33:02.312 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 3727, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1477524, 2024-09-05T01:33:02.313 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 3727, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3727, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1477524, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:02.314 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3373, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 33815, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3373, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 33815, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.315 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 3373, 2024-09-05T01:33:02.316 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 3373, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 3373, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 3373, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:02.317 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:02.318 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x556108966b40": { 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:02.319 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:02.320 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-05T01:33:02.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:02 smithi183 ceph-mon[34589]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-09-05T01:33:02.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:02 smithi129 ceph-mon[30979]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-09-05T01:33:02.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:02 smithi194 ceph-mon[34941]: pgmap v884: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 12 KiB/s wr, 21 op/s 2024-09-05T01:33:03.151 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-05T01:33:03.151 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-05T01:33:03.151 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-05T01:33:03.151 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-05T01:33:03.151 DEBUG:tasks.check_counter:Getting stats from h 2024-09-05T01:33:03.152 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-05T01:33:03.411 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:33:03.944 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 6595, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 3312, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1877824, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 863103, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 12, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.317015536, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.141955586, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.232118546, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.005067136, 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:03.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3310, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.107880121, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000032592 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9370, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001550008, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000165 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.946 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 63, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 309, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 15578, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 630703, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 15, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.021010450, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.011348719, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.007475259, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000231416, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 307, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.024086020, 2024-09-05T01:33:03.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000078456 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 99, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000016495, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000166 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 1904, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 594384 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 402, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 381, 2024-09-05T01:33:03.948 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 864491, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 243261, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 11, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 5, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.051016911, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020823850, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.038457372, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000539647, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 381, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.031392542, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000082395 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.949 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000156, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000078 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 861440, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 208192 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:03.950 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 42, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001879938, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000044760 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000006757, 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000003378 2024-09-05T01:33:03.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 29, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 4, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001737471, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000434367 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 25, 2024-09-05T01:33:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 11, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 34, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 9, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 25, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 18, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 4, 2024-09-05T01:33:03.953 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 3, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 3, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 137, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 88, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 24, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 24, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-09-05T01:33:03.954 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 1013, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 1450841979, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:03.955 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 3, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 3, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-09-05T01:33:03.956 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 1 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 119, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 71, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 79, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-09-05T01:33:03.957 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 17, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 372, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-09-05T01:33:03.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 5, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 5, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 2, 2024-09-05T01:33:03.959 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 2, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 4195905, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.007660219, 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001915054 2024-09-05T01:33:03.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 38, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 40, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 2, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 56, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 65, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 9, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 34, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 38, 2024-09-05T01:33:03.961 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 4, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 3, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 4, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 1, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53700, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 223520 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 33, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 29, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 17, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 1, 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-09-05T01:33:03.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 4, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001737471, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000434367 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:03.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.965 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.966 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-09-05T01:33:03.967 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.968 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.969 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.970 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.972 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.973 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-09-05T01:33:03.974 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 3, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 3, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 1056, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:03.975 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:03.976 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:03.977 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 254951, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 85, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:03.978 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 174, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 184029, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 612, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:03.979 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 40, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 11972, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 33, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 40, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 40, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.232997572, 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.005824939 2024-09-05T01:33:03.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 40, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 63 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 40, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 0, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 40, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-09-05T01:33:03.981 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 4, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 6, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 6, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 0, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-09-05T01:33:03.982 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-09-05T01:33:03.983 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 93, 2024-09-05T01:33:03.984 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 7, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 10, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 17, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 7, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 16, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-09-05T01:33:03.985 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "osd-slow-ops": { 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 274 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-09-05T01:33:03.986 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 7056, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 2099942, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 7056, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 7056, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2099942, 2024-09-05T01:33:03.987 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:03.988 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 40, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 11972, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 40, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 11972, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:03.989 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 40, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 40, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 40, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 40, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:03.991 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x564cd8d7c960": { 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 5, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1601, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 5, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:03.992 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 4, 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1601, 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:03.993 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-05T01:33:04.191 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:04 smithi183 ceph-mon[34589]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-05T01:33:04.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:04 smithi129 ceph-mon[30979]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-05T01:33:04.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:04 smithi194 ceph-mon[34941]: pgmap v885: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 11 KiB/s wr, 16 op/s 2024-09-05T01:33:04.832 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-05T01:33:04.832 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-05T01:33:04.833 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-05T01:33:04.833 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=119 2024-09-05T01:33:04.833 DEBUG:tasks.check_counter:Getting stats from k 2024-09-05T01:33:04.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-05T01:33:05.092 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T01:33:05.610 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 338, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 331, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 789456, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 203888, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.046813071, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.020071316, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.032307830, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:05.611 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 331, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.020226318, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000061106 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 789456, 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 203888 2024-09-05T01:33:05.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 2, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 252, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 320, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 201792, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.014150807, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.010461319, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.001222690, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:05.613 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 250, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.012569555, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000050278 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 201792 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.614 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 0, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 2, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 0, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 0, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.000383964, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000279857, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:05.615 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.616 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:05.617 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:05.618 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:05.619 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 148704, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 29, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:05.620 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 336, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 739220, 2024-09-05T01:33:05.621 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 336, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 336, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 739220, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-09-05T01:33:05.622 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-09-05T01:33:06.272 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:06 smithi183 ceph-mon[34589]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-05T01:33:06.304 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-05T01:33:06.304 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-05T01:33:06.305 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-05T01:33:06.305 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-05T01:33:06.305 DEBUG:tasks.check_counter:Getting stats from c 2024-09-05T01:33:06.305 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-05T01:33:06.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:06 smithi129 ceph-mon[30979]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-05T01:33:06.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:06 smithi194 ceph-mon[34941]: pgmap v886: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.9 KiB/s wr, 14 op/s 2024-09-05T01:33:06.601 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:33:07.324 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 705, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 705, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 3764825, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 223376, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.069790302, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.021414469, 2024-09-05T01:33:07.325 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.021051708, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.020933555, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 704, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.034492105, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000048994 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:33:07.326 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 633, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 714, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 8502343, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 429337, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.053697525, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.016815031, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.017659295, 2024-09-05T01:33:07.327 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.014897393, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 712, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.029080798, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000040843 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.328 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 228304 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 118, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 117, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 187187, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 67764, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.015452496, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.007010355, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.009727741, 2024-09-05T01:33:07.329 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000097255, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 116, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.010455632, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000090134 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 185792, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 66800 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.330 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2237, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.062001281, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000027716 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:07.331 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.332 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 415, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 154, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 261, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 27, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-09-05T01:33:07.333 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 4, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-09-05T01:33:07.334 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 1013, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 1450841979, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:07.335 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-09-05T01:33:07.336 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-09-05T01:33:07.337 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:07.338 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 4950, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-09-05T01:33:07.339 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 29, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 45461174, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 45494728, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 45490161, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 4950 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.340 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 151, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 246, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 95, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 36, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 36, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 415, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 415, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-09-05T01:33:07.341 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 48756, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215328 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-09-05T01:33:07.342 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.343 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.344 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:07.345 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-09-05T01:33:07.346 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.347 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.348 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.349 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.350 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.351 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.352 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-09-05T01:33:07.353 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 2129, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:07.354 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:07.355 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:07.356 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 198367, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 54, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:07.357 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 174, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 669786, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 11018, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-09-05T01:33:07.358 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 1336, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 26, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 4, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 1336, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1336, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.957980743, 2024-09-05T01:33:07.359 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000717051 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1336, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 1337 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 1336, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 1336, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 782, 2024-09-05T01:33:07.360 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 551, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:07.361 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 2, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-09-05T01:33:07.362 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-09-05T01:33:07.363 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 6, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 6, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-09-05T01:33:07.364 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-09-05T01:33:07.365 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 1452, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 12314717, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 1452, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:07.366 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1452, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 12314717, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:07.367 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 1336, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 11892954, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1336, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 11892954, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:07.368 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 1336, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1336, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 1336, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1336, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.369 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:07.370 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55940c6046e0": { 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:07.371 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:07.372 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-09-05T01:33:08.391 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:08 smithi194 ceph-mon[34941]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-05T01:33:08.403 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-05T01:33:08.404 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-05T01:33:08.404 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-05T01:33:08.404 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-05T01:33:08.404 DEBUG:tasks.check_counter:Getting stats from f 2024-09-05T01:33:08.404 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-05T01:33:08.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:08 smithi183 ceph-mon[34589]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-05T01:33:08.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:08 smithi129 ceph-mon[30979]: pgmap v887: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-05T01:33:08.661 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:33:09.261 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-09-05T01:33:09.261 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:09.261 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 1515, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 1498, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 1236736, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 572598, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 1, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.168114651, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.057200596, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.067722380, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.039175402, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 1498, 2024-09-05T01:33:09.262 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.078892723, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000052665 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 847984, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 204672 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2143, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 2390, 2024-09-05T01:33:09.263 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 544958, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1275494, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 6, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.140766371, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.047977403, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.034045561, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.048808654, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2388, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.086916082, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000036397 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.264 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 594912 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 94, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 96, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 26174, 2024-09-05T01:33:09.265 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 29899, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 5, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 3, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.009859947, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.003349547, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.003297531, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.002217585, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 94, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.005190904, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000055222 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.266 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-09-05T01:33:09.267 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 5676, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.137030384, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000024142 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.268 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:09.269 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 1, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 0, 2024-09-05T01:33:09.270 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-09-05T01:33:09.271 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:09.272 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-09-05T01:33:09.273 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-09-05T01:33:09.274 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.275 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 5, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-09-05T01:33:09.276 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4195905, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 4195426, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 5 2024-09-05T01:33:09.277 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 3, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 3, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 1, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 1, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 0, 2024-09-05T01:33:09.278 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 50916, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 215328 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.279 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.280 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.281 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.282 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.283 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-09-05T01:33:09.284 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.285 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.286 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-09-05T01:33:09.287 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.288 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.289 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.290 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 2, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 1, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 0, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 0, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 2130, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-09-05T01:33:09.291 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:09.292 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:09.293 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 190167, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 52, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:09.294 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 174, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 8512, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 13, 2024-09-05T01:33:09.295 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 3391, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 21, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 3, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 3391, 2024-09-05T01:33:09.296 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3391, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 6.059876434, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001787047 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3391, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 3392 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 3391, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 3391, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 0, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-09-05T01:33:09.297 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 2225, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 1164, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:09.298 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 1, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-09-05T01:33:09.299 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-09-05T01:33:09.300 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 93, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 7, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 5, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 6, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 1, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 0, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 2, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 0 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.301 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 0 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-09-05T01:33:09.302 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 3748, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 1449358, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 3748, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:09.303 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3748, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1449358, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:09.304 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3391, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 1606, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3391, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 1606, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:09.305 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 3391, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 3391, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 3391, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 3391, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.306 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:09.307 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x559a25300b40": { 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:09.308 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.309 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:09.310 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-09-05T01:33:10.313 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-05T01:33:10.313 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-05T01:33:10.313 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-05T01:33:10.313 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-05T01:33:10.313 DEBUG:tasks.check_counter:Getting stats from i 2024-09-05T01:33:10.313 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-05T01:33:10.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:10 smithi183 ceph-mon[34589]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-05T01:33:10.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:10 smithi129 ceph-mon[30979]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-05T01:33:10.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:10 smithi194 ceph-mon[34941]: pgmap v888: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-09-05T01:33:10.586 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 106, 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 103, 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 29686, 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 114642, 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 13, 2024-09-05T01:33:11.172 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.013044413, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.002364503, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.008841974, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000913090, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 102, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.019787312, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000193993 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 126, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000021550, 2024-09-05T01:33:11.173 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000171 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 37, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 286, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 20455, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 653409, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 12, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 5, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.018141207, 2024-09-05T01:33:11.174 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.010708349, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.004725048, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000507316, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 284, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.027564382, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000097057 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 27, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000004064, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000150 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:33:11.175 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 596352 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 6955, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 3652, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 2719140, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 1011854, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 7, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 4, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.321802506, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.123229859, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.269069539, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.006114274, 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:11.176 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3651, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.095923430, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000026273 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 9460, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001655909, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000175 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 853888, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 207584 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 1, 2024-09-05T01:33:11.177 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-MDSRank": { 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 70, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003804474, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000054349 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "finisher-PurgeQueue": { 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "queue_len": 0, 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "complete_latency": { 2024-09-05T01:33:11.178 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000013068, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000006534 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "mds": { 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "request": 34, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "reply": 10, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "reply_latency": { 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 10, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.004116590, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000411659 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_reply": 0, 2024-09-05T01:33:11.179 INFO:teuthology.orchestra.run.smithi194.stdout: "forward": 24, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_complete": 0, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_fetch_keys": 0, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_commit": 11, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_split": 0, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_merge": 0, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes": 36, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_top": 9, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_bottom": 27, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pin_tail": 0, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_pinned": 22, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_expired": 4, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "inodes_with_caps": 4, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "caps": 4, 2024-09-05T01:33:11.180 INFO:teuthology.orchestra.run.smithi194.stdout: "subtrees": 5, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse": 135, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_hit": 88, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_forward": 24, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_discover": 23, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_dir_fetch": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_remote_ino": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "traverse_lock": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "load_cent": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "q": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "exported": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "exported_inodes": 0, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "imported": 2, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "imported_inodes": 2, 2024-09-05T01:33:11.181 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_dir_fetch": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_backtrace_fetch": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "openino_peer_discover": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rfiles": 1013, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rbytes": 1450841979, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "root_rsnaps": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_set_tag": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_inotable_repaired": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_inodes": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "scrub_file_inodes": 0, 2024-09-05T01:33:11.182 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_inode_file_caps": 0, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_revoke": 6, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_grant": 2, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps": 16, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_caps_dirty": 3, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_cap_release": 0, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "process_request_cap_release": 0 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_cache": { 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update": 6, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_update_receipt": 104, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_try_discover": 62, 2024-09-05T01:33:11.183 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_send_discover": 65, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "dir_handle_discover": 6, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_delayed": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_strays_enqueuing": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_created": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_enqueued": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_reintegrated": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "strays_migrated": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_processing": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_enqueued": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "num_recovering_prioritized": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_started": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "recovery_completed": 0, 2024-09-05T01:33:11.184 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_path": 17, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_quiesce_inode": 406, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_exportdir": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_flush": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragmentdir": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_fragstats": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "ireq_inodestats": 0 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_log": { 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "evlrg": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "evadd": 22, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "evex": 0, 2024-09-05T01:33:11.185 INFO:teuthology.orchestra.run.smithi194.stdout: "evtrm": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "ev": 22, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evexg": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "evexd": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segadd": 2, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segex": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segtrm": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "seg": 2, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segmjr": 2, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segexg": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "segexd": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "expos": 4194304, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "wrpos": 4228114, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "rdpos": 0, 2024-09-05T01:33:11.186 INFO:teuthology.orchestra.run.smithi194.stdout: "jlat": { 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 16, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.029252753, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001828297 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "replayed": 0 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_mem": { 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "ino": 40, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "ino+": 42, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "ino-": 2, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "dir": 58, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "dir+": 62, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "dir-": 4, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "dn": 36, 2024-09-05T01:33:11.187 INFO:teuthology.orchestra.run.smithi194.stdout: "dn+": 40, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dn-": 4, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "cap": 4, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "cap+": 4, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "cap-": 0, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "rss": 52392, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "heap": 223520 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_server": { 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_client_request": 34, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "dispatch_server_request": 0, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_request": 34, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_client_session": 13, 2024-09-05T01:33:11.188 INFO:teuthology.orchestra.run.smithi194.stdout: "handle_peer_request": 0, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "req_create_latency": { 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getattr_latency": { 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getfilelock_latency": { 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.189 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_link_latency": { 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookup_latency": { 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 2, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000880948, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000440474 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookuphash_latency": { 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.190 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupino_latency": { 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupname_latency": { 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupparent_latency": { 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.191 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lookupsnap_latency": { 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_lssnap_latency": { 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mkdir_latency": { 2024-09-05T01:33:11.192 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mknod_latency": { 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "req_mksnap_latency": { 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.193 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "req_open_latency": { 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001724718, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000431179 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "req_readdir_latency": { 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 4, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.001510924, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000377731 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rename_latency": { 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.194 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_renamesnap_latency": { 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_snapdiff_latency": { 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmdir_latency": { 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.195 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmsnap_latency": { 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_rmxattr_latency": { 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setattr_latency": { 2024-09-05T01:33:11.196 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setdirlayout_latency": { 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setfilelock_latency": { 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.197 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setlayout_latency": { 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "req_setxattr_latency": { 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "req_symlink_latency": { 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.198 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "req_unlink_latency": { 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_revoke_eviction": 0, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "cap_acquisition_throttle": 0, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "req_getvxattr_latency": { 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.199 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_sessions": { 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "session_count": 1, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "session_add": 1, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "session_remove": 0, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_open": 1, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "sessions_stale": 0, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "total_load": 2, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "average_load": 2, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "avg_session_uptime": 1063, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "mdthresh_evicted": 0 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:11.200 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:11.201 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:11.202 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 262743, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 82, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-09-05T01:33:11.203 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 9240, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 174, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 193205, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 677, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:11.204 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "objecter": { 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_active": 0, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_laggy": 0, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send": 60, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_send_bytes": 45482, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_resend": 28, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_reply": 60, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "op_latency": { 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 60, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.237995049, 2024-09-05T01:33:11.205 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.003966584 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_inflight": 0, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "oplen_avg": { 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 60, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 88 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op": 60, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_r": 0, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_w": 60, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_rmw": 0, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "op_pg": 0, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_stat": 0, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_create": 10, 2024-09-05T01:33:11.206 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_read": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_write": 16, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writefull": 9, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_writesame": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_append": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_zero": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_truncate": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_delete": 6, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_mapext": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_sparse_read": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_clonerange": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_getxattr": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_setxattr": 20, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_cmpxattr": 0, 2024-09-05T01:33:11.207 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_rmxattr": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_resetxattrs": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_call": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_watch": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_notify": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_pgls_filter": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "osdop_other": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_active": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_send": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_resend": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "linger_ping": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_active": 0, 2024-09-05T01:33:11.208 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_send": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolop_resend": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_active": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_send": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "poolstat_resend": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_active": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_send": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "statfs_resend": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "command_active": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "command_send": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "command_resend": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "map_epoch": 93, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "map_full": 0, 2024-09-05T01:33:11.209 INFO:teuthology.orchestra.run.smithi194.stdout: "map_inc": 7, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_sessions": 8, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_open": 12, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_session_close": 4, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_laggy": 0, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_wr": 23, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_rd": 0, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_del": 4 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "oft": { 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_objs": 1, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_kv_pairs": 0, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_updates": 0, 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: "omap_total_removes": 0 2024-09-05T01:33:11.210 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "osd-slow-ops": { 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "slow_ops_count": 298 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "purge_queue": { 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executing_high_water": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed_ops": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_executed": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "pq_item_in_journal": 0 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:11.211 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 7094, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 2108437, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 7094, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 7094, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 2108437, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.212 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_bytes": { 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 60, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 45482, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 60, 2024-09-05T01:33:11.213 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 45482, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-objecter_ops": { 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 1024, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:11.214 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 60, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 60, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 60, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 60, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle": { 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:11.215 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.216 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-write_buf_throttle-0x55e45c1b2960": { 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 3758096384, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 22, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 33810, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 22, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 16, 2024-09-05T01:33:11.217 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 33810, 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:11.218 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-09-05T01:33:12.031 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-05T01:33:12.031 INFO:tasks.check_counter:Daemon mds.i mds.imported=2 2024-09-05T01:33:12.031 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=6 2024-09-05T01:33:12.031 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=104 2024-09-05T01:33:12.031 DEBUG:tasks.check_counter:Getting stats from l 2024-09-05T01:33:12.031 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-05T01:33:12.282 INFO:teuthology.orchestra.run.smithi194.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T01:33:12.403 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:12 smithi194 ceph-mon[34941]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:12.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:12 smithi183 ceph-mon[34589]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:12.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:12 smithi129 ceph-mon[30979]: pgmap v889: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout:{ 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 336, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 331, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 786960, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 204720, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 0, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.046751343, 2024-09-05T01:33:12.800 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.020466765, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.032194119, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 330, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.019261166, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000058367 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.801 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 786960, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 204720 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 2, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 254, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 320, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 202144, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 4, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 2, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.013555779, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.010141731, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.000984329, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:12.802 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 252, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.012103930, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000048031 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 202144 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.803 INFO:teuthology.orchestra.run.smithi194.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_messages": 4, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages": 4, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_bytes": 3040, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_bytes": 448, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_created_connections": 2, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_active_connections": 1, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_total_time": 0.002173563, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_send_time": 0.000036293, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_recv_time": 0.002105883, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 3, 2024-09-05T01:33:12.804 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.003896996, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.001298998 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_handle_ack_lat": { 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "cct": { 2024-09-05T01:33:12.805 INFO:teuthology.orchestra.run.smithi194.stdout: "total_workers": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "unhealthy_workers": 0 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "mempool": { 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_bytes": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bloom_filter_items": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_alloc_items": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_data_items": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T01:33:12.806 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_other_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_extent_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_blob_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_fsck_items": 0, 2024-09-05T01:33:12.807 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_txc_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluestore_writing_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_reader_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "bluefs_file_writer_items": 0, 2024-09-05T01:33:12.808 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_bytes": 148744, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_anon_items": 30, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "buffer_meta_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_mapbl_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osd_pglog_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "osdmap_mapping_items": 0, 2024-09-05T01:33:12.809 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_bytes": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "pgmap_items": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_bytes": 928, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "mds_co_items": 2, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_bytes": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_1_items": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_bytes": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "unittest_2_items": 0 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: }, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "val": 0, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "max": 104857600, 2024-09-05T01:33:12.810 INFO:teuthology.orchestra.run.smithi194.stdout: "get_started": 0, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get": 338, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get_sum": 739480, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_fail": 0, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "get_or_fail_success": 338, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "take": 0, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "take_sum": 0, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "put": 338, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "put_sum": 739480, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "wait": { 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgcount": 0, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "sum": 0.000000000, 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: "avgtime": 0.000000000 2024-09-05T01:33:12.811 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:12.812 INFO:teuthology.orchestra.run.smithi194.stdout: } 2024-09-05T01:33:12.812 INFO:teuthology.orchestra.run.smithi194.stdout:} 2024-09-05T01:33:13.453 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:13 smithi194 ceph-mon[34941]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:13.455 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-05T01:33:13.455 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-05T01:33:13.455 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-05T01:33:13.455 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-05T01:33:13.455 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-05T01:33:13.465 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-05T01:33:13.465 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:33:13.465 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T01:33:13.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:13 smithi129 ceph-mon[30979]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:13.496 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:33:13.496 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T01:33:13.556 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T01:33:13.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:13 smithi183 ceph-mon[34589]: pgmap v890: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:13.852 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- 172.21.15.129:0/2594525511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 msgr2=0x7f481410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 --2- 172.21.15.129:0/2594525511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481410da40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f480c009ca0 tx=0x7f480c02a950 comp rx=0 tx=0).stop 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- 172.21.15.129:0/2594525511 shutdown_connections 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 --2- 172.21.15.129:0/2594525511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481410da40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 --2- 172.21.15.129:0/2594525511 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f48141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 --2- 172.21.15.129:0/2594525511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 0x7f4814101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- 172.21.15.129:0/2594525511 >> 172.21.15.129:0/2594525511 conn(0x7f48140fc7d0 msgr2=0x7f48140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- 172.21.15.129:0/2594525511 shutdown_connections 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- 172.21.15.129:0/2594525511 wait complete. 2024-09-05T01:33:14.348 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 Processor -- start 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.345+0000 7f4819b57640 1 -- start start 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 0x7f48141972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f4814197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4814110e30 con 0x7f4814102ef0 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4814110cb0 con 0x7f4814100e20 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4819b57640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f4814110fb0 con 0x7f4814102570 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4813fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f4814197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4813fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55634/0 (socket says 172.21.15.129:55634) 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4813fff640 1 -- 172.21.15.129:0/271302359 learned_addr learned my addr 172.21.15.129:0/271302359 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:14.349 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f48137fe640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 0x7f48141972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 -- 172.21.15.129:0/271302359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 msgr2=0x7f48141972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 0x7f48141972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 -- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 msgr2=0x7f481419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 -- 172.21.15.129:0/271302359 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481419c280 con 0x7f4814102570 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f4813fff640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.346+0000 7f480bfff640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f4814197810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00d6d0 tx=0x7f47fc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.347+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc017070 con 0x7f4814102570 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.347+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47fc009820 con 0x7f4814102570 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.347+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481419c570 con 0x7f4814102570 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.347+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47fc004c40 con 0x7f4814102570 2024-09-05T01:33:14.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.347+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141a3e50 con 0x7f4814102570 2024-09-05T01:33:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.348+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4814101200 con 0x7f4814102570 2024-09-05T01:33:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.349+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f47fc0099d0 con 0x7f4814102570 2024-09-05T01:33:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.349+0000 7f48117fa640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:14.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.349+0000 7f48137fe640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.350+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f47fc0cecc0 con 0x7f4814102570 2024-09-05T01:33:14.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.350+0000 7f48137fe640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f480400bbe0 tx=0x7f480400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:14.354 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.354+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47fc09b1a0 con 0x7f4814102570 2024-09-05T01:33:14.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.539+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4814109090 con 0x7f4814102570 2024-09-05T01:33:14.540 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.539+0000 7f48117fa640 1 -- 172.21.15.129:0/271302359 <== mon.1 v2:172.21.15.194:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+601 (secure 0 0 0) 0x7f47fc09f4a0 con 0x7f4814102570 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/665628602 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3304384691 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/2755277182 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4057892175 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4214690776 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:14.541 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3403026756 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:listed 10 entries 2024-09-05T01:33:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.541+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 msgr2=0x7f48000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:14.542 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.541+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f480400bbe0 tx=0x7f480400b390 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.541+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 msgr2=0x7f4814197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.541+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f4814197810 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f47fc00d6d0 tx=0x7f47fc00db90 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 shutdown_connections 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f48000a2b80 0x7f48000a5040 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4814102ef0 0x7f481419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f4814102570 0x7f4814197810 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 --2- 172.21.15.129:0/271302359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4814100e20 0x7f48141972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 >> 172.21.15.129:0/271302359 conn(0x7f48140fc7d0 msgr2=0x7f48140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 shutdown_connections 2024-09-05T01:33:14.543 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:14.542+0000 7f4819b57640 1 -- 172.21.15.129:0/271302359 wait complete. 2024-09-05T01:33:15.355 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-05T01:33:15.355 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T01:33:15.374 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T01:33:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:15 smithi129 ceph-mon[30979]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:15.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:15 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/271302359' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:15.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:15 smithi194 ceph-mon[34941]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:15.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:15 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/271302359' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:15.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:15 smithi183 ceph-mon[34589]: pgmap v891: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 6 op/s 2024-09-05T01:33:15.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:15 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/271302359' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:15.674 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 -- 172.21.15.129:0/2524722639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 msgr2=0x7fb23c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 --2- 172.21.15.129:0/2524722639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c10ea90 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb234009ca0 tx=0x7fb23402a8c0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 -- 172.21.15.129:0/2524722639 shutdown_connections 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 --2- 172.21.15.129:0/2524722639 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c10ea90 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 --2- 172.21.15.129:0/2524722639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c102790 0x7fb23c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 --2- 172.21.15.129:0/2524722639 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c101de0 0x7fb23c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 -- 172.21.15.129:0/2524722639 >> 172.21.15.129:0/2524722639 conn(0x7fb23c0fd8c0 msgr2=0x7fb23c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 -- 172.21.15.129:0/2524722639 shutdown_connections 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.152+0000 7fb240b6a640 1 -- 172.21.15.129:0/2524722639 wait complete. 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 Processor -- start 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 -- start start 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c101de0 0x7fb23c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c102790 0x7fb23c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111e40 con 0x7fb23c106f10 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111cc0 con 0x7fb23c101de0 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb240b6a640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111fc0 con 0x7fb23c102790 2024-09-05T01:33:16.174 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb23ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb23ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55654/0 (socket says 172.21.15.129:55654) 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb23ad9d640 1 -- 172.21.15.129:0/1253011436 learned_addr learned my addr 172.21.15.129:0/1253011436 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb23a59c640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c101de0 0x7fb23c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.153+0000 7fb233fff640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c102790 0x7fb23c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 -- 172.21.15.129:0/1253011436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c102790 msgr2=0x7fb23c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c102790 0x7fb23c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 -- 172.21.15.129:0/1253011436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c101de0 msgr2=0x7fb23c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c101de0 0x7fb23c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 -- 172.21.15.129:0/1253011436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c19d370 con 0x7fb23c106f10 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb23ad9d640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb234009dc0 tx=0x7fb234032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234003cf0 con 0x7fb23c106f10 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2340044f0 con 0x7fb23c106f10 2024-09-05T01:33:16.175 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c19d600 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb234030b80 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.154+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a4f40 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.155+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c1021c0 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.156+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fb2340046a0 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.157+0000 7fb2337fe640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 0x7fb2280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.157+0000 7fb23a59c640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 0x7fb2280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.157+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fb2340f08e0 con 0x7fb23c106f10 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.157+0000 7fb23a59c640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 0x7fb2280a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009990 tx=0x7fb22c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:16.176 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.161+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2340bd430 con 0x7fb23c106f10 2024-09-05T01:33:16.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.339+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb23c109b60 con 0x7fb23c106f10 2024-09-05T01:33:16.340 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.339+0000 7fb2337fe640 1 -- 172.21.15.129:0/1253011436 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v93) ==== 81+0+601 (secure 0 0 0) 0x7fb2340bcd30 con 0x7fb23c106f10 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3942407634 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/665628602 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3304384691 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6800/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/2755277182 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4057892175 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/4214690776 2024-09-06T01:06:17.664286+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:6801/3490891928 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:16.341 INFO:teuthology.orchestra.run.smithi129.stdout:172.21.15.129:0/3403026756 2024-09-06T01:05:54.362223+0000 2024-09-05T01:33:16.342 INFO:teuthology.orchestra.run.smithi129.stderr:listed 10 entries 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 msgr2=0x7fb2280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 0x7fb2280a50c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009990 tx=0x7fb22c009110 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 msgr2=0x7fb23c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb234009dc0 tx=0x7fb234032ff0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 shutdown_connections 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fb2280a2c00 0x7fb2280a50c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb23c106f10 0x7fb23c19cc90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fb23c102790 0x7fb23c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 --2- 172.21.15.129:0/1253011436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb23c101de0 0x7fb23c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 >> 172.21.15.129:0/1253011436 conn(0x7fb23c0fd8c0 msgr2=0x7fb23c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:16.343 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 shutdown_connections 2024-09-05T01:33:16.344 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:16.342+0000 7fb240b6a640 1 -- 172.21.15.129:0/1253011436 wait complete. 2024-09-05T01:33:17.100 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi129.front.sepia.ceph.com... 2024-09-05T01:33:17.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:33:17.101 DEBUG:teuthology.orchestra.run.smithi129:> sudo fusermount -u /tmp/cephfs 2024-09-05T01:33:17.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi129.stderr:ceph-fuse[84335]: fuse finished with error 0 and tester_r 0 2024-09-05T01:33:17.292 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T01:33:17.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:17 smithi129 ceph-mon[30979]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:17.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:17 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1253011436' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:17.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:17 smithi194 ceph-mon[34941]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:17.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:17 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1253011436' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:17.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:17 smithi183 ceph-mon[34589]: pgmap v892: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-05T01:33:17.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:17 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1253011436' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T01:33:19.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:19 smithi129 ceph-mon[30979]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-05T01:33:19.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:19 smithi194 ceph-mon[34941]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-05T01:33:19.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:19 smithi183 ceph-mon[34589]: pgmap v893: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-05T01:33:21.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:21 smithi129 ceph-mon[30979]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:21.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:21 smithi194 ceph-mon[34941]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:21.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:21 smithi183 ceph-mon[34589]: pgmap v894: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:23.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:33:23.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:33:23.394 DEBUG:teuthology.orchestra.run.smithi129:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-05T01:33:23.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:33:23.425 DEBUG:teuthology.orchestra.run.smithi129:> ip netns list 2024-09-05T01:33:23.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:23 smithi129 ceph-mon[30979]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:23.495 INFO:teuthology.orchestra.run.smithi129.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-05T01:33:23.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:33:23.495 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-05T01:33:23.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:23 smithi194 ceph-mon[34941]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:23.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T01:33:23.566 DEBUG:teuthology.orchestra.run.smithi129:> sudo ip link delete ceph-brx 2024-09-05T01:33:23.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:23 smithi183 ceph-mon[34589]: pgmap v895: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:23.663 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-05T01:33:23.674 INFO:tasks.cephadm:Teardown begin 2024-09-05T01:33:23.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-05T01:33:23.674 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:33:23.931 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- 172.21.15.129:0/3949959535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01027a0 msgr2=0x7f7ed01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/3949959535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01027a0 0x7f7ed01067f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009900 tx=0x7f7eb802a3f0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.426 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- 172.21.15.129:0/3949959535 shutdown_connections 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/3949959535 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed0106f20 0x7f7ed010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/3949959535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed01027a0 0x7f7ed01067f0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/3949959535 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0101df0 0x7f7ed01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- 172.21.15.129:0/3949959535 >> 172.21.15.129:0/3949959535 conn(0x7f7ed00fd8f0 msgr2=0x7f7ed00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- 172.21.15.129:0/3949959535 shutdown_connections 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- 172.21.15.129:0/3949959535 wait complete. 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 Processor -- start 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.397+0000 7f7ed68e0640 1 -- start start 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed01027a0 0x7f7ed0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 0x7f7ed019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111e30 con 0x7f7ed0101df0 2024-09-05T01:33:24.427 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111cb0 con 0x7f7ed0106f20 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed0111fb0 con 0x7f7ed01027a0 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46682/0 (socket says 172.21.15.129:46682) 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 -- 172.21.15.129:0/2171822092 learned_addr learned my addr 172.21.15.129:0/2171822092 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ec7fff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed01027a0 0x7f7ed0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 -- 172.21.15.129:0/2171822092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed01027a0 msgr2=0x7f7ed0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed4e7d640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 0x7f7ed019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed01027a0 0x7f7ed0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 -- 172.21.15.129:0/2171822092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 msgr2=0x7f7ed019cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 0x7f7ed019cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 -- 172.21.15.129:0/2171822092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed019d240 con 0x7f7ed0101df0 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed4e7d640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 0x7f7ed019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:33:24.428 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecffff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c920 tx=0x7f7ec000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0007c70 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ec0007e10 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed019d530 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0018820 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.398+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed01a4e10 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.399+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed01021d0 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.401+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f7ec00189c0 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.401+0000 7f7ecdffb640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.401+0000 7f7ec7fff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.401+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f7ec00cf960 con 0x7f7ed0101df0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.402+0000 7f7ec7fff640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8031440 tx=0x7f7eb80095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:24.429 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.405+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ec0014030 con 0x7f7ed0101df0 2024-09-05T01:33:24.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.605+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed0199690 con 0x7f7ed0101df0 2024-09-05T01:33:24.606 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.606+0000 7f7ecdffb640 1 -- 172.21.15.129:0/2171822092 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f7ec009c4e0 con 0x7f7ed0101df0 2024-09-05T01:33:24.607 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:33:24.607 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:33:24.607 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 msgr2=0x7f7ebc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8031440 tx=0x7f7eb80095f0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 msgr2=0x7f7ed0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000c920 tx=0x7f7ec000cde0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 shutdown_connections 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f7ebc0a2b80 0x7f7ebc0a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ed0106f20 0x7f7ed019cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7ed01027a0 0x7f7ed0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 --2- 172.21.15.129:0/2171822092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7ed0101df0 0x7f7ed0198330 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:24.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 >> 172.21.15.129:0/2171822092 conn(0x7f7ed00fd8f0 msgr2=0x7f7ed00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:24.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 shutdown_connections 2024-09-05T01:33:24.610 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:24.608+0000 7f7ed68e0640 1 -- 172.21.15.129:0/2171822092 wait complete. 2024-09-05T01:33:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:25 smithi129 ceph-mon[30979]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T01:33:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:25 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/2171822092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:25.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:25 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:33:25.327 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T01:33:25.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:25 smithi194 ceph-mon[34941]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T01:33:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:25 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/2171822092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:25.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:25 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:33:25.596 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:25.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:25 smithi183 ceph-mon[34589]: pgmap v896: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-09-05T01:33:25.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:25 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/2171822092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:25.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:25 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 -- 172.21.15.129:0/2765013233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120102d90 msgr2=0x7f7120103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 --2- 172.21.15.129:0/2765013233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120102d90 0x7f7120103170 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f711c009930 tx=0x7f711c02a420 comp rx=0 tx=0).stop 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 -- 172.21.15.129:0/2765013233 shutdown_connections 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 --2- 172.21.15.129:0/2765013233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120108290 0x7f712010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 --2- 172.21.15.129:0/2765013233 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120103740 0x7f7120107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.078 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 --2- 172.21.15.129:0/2765013233 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120102d90 0x7f7120103170 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 -- 172.21.15.129:0/2765013233 >> 172.21.15.129:0/2765013233 conn(0x7f71200fc7d0 msgr2=0x7f71200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 -- 172.21.15.129:0/2765013233 shutdown_connections 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.050+0000 7f712774f640 1 -- 172.21.15.129:0/2765013233 wait complete. 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 Processor -- start 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 -- start start 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 0x7f7120198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 0x7f712019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111f40 con 0x7f7120103740 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111dc0 con 0x7f7120102d90 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f712774f640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f71201120c0 con 0x7f7120108290 2024-09-05T01:33:26.079 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f7124cea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f7124cea640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46696/0 (socket says 172.21.15.129:46696) 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f7124cea640 1 -- 172.21.15.129:0/3184368087 learned_addr learned my addr 172.21.15.129:0/3184368087 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f71254eb640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 0x7f7120198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f7125cec640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 0x7f712019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.051+0000 7f7124cea640 1 -- 172.21.15.129:0/3184368087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 msgr2=0x7f712019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7124cea640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 0x7f712019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7124cea640 1 -- 172.21.15.129:0/3184368087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 msgr2=0x7f7120198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7124cea640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 0x7f7120198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7124cea640 1 -- 172.21.15.129:0/3184368087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712019d4b0 con 0x7f7120103740 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f71254eb640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 0x7f7120198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7125cec640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 0x7f712019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:33:26.080 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f7124cea640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f711000c970 tx=0x7f711000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110007bb0 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7110007d50 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712019d7a0 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110018730 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.052+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a5080 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.053+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120103170 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.054+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f711000f450 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.055+0000 7f71167fc640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 0x7f710c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.055+0000 7f71254eb640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 0x7f710c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.055+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f71100cf5d0 con 0x7f7120103740 2024-09-05T01:33:26.081 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.055+0000 7f71254eb640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 0x7f710c0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:26.082 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.059+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7110014030 con 0x7f7120103740 2024-09-05T01:33:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:33:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:33:26.242 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:26 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:33:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.266+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f712010a2d0 con 0x7f7120103740 2024-09-05T01:33:26.268 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.267+0000 7f71167fc640 1 -- 172.21.15.129:0/3184368087 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11930 (secure 0 0 0) 0x7f711009ba70 con 0x7f7120103740 2024-09-05T01:33:26.269 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:33:26.269 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":34,"btime":"2024-09-05T01:26:35:144547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/2733798787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":2733798787},{"type":"v1","addr":"172.21.15.129:6835","nonce":2733798787}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/1098380818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":1098380818},{"type":"v1","addr":"172.21.15.129:6837","nonce":1098380818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14610,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2120045366","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2120045366},{"type":"v1","addr":"172.21.15.129:6839","nonce":2120045366}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6841/3095518263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6840","nonce":3095518263},{"type":"v1","addr":"172.21.15.129:6841","nonce":3095518263}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.194:6833/1551944445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6832","nonce":1551944445},{"type":"v1","addr":"172.21.15.194:6833","nonce":1551944445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24404,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/263217116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":263217116},{"type":"v1","addr":"172.21.15.183:6833","nonce":263217116}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T01:13:06.500509+0000","modified":"2024-09-05T01:26:35.144524+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24452,"mds_2":24451},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/3903364483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":3903364483},{"type":"v1","addr":"172.21.15.183:6835","nonce":3903364483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6835/1641200590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6834","nonce":1641200590},{"type":"v1","addr":"172.21.15.194:6835","nonce":1641200590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.194:6839/203556383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6838","nonce":203556383},{"type":"v1","addr":"172.21.15.194:6839","nonce":203556383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.183:6837/685319691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":685319691},{"type":"v1","addr":"172.21.15.183:6837","nonce":685319691}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/3533858614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":3533858614},{"type":"v1","addr":"172.21.15.183:6839","nonce":3533858614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25945":{"gid":25945,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.194:6837/725078520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6836","nonce":725078520},{"type":"v1","addr":"172.21.15.194:6837","nonce":725078520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24488,"qdb_cluster":[24488,24452,24451]},"id":1}]} 2024-09-05T01:33:26.270 INFO:teuthology.orchestra.run.smithi129.stderr:dumped fsmap epoch 34 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 msgr2=0x7f710c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 0x7f710c0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c031400 comp rx=0 tx=0).stop 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 msgr2=0x7f7120198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f711000c970 tx=0x7f711000ce30 comp rx=0 tx=0).stop 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 shutdown_connections 2024-09-05T01:33:26.271 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f710c0a2c00 0x7f710c0a50c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f7120108290 0x7f712019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7120103740 0x7f7120198a40 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 --2- 172.21.15.129:0/3184368087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7120102d90 0x7f7120198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 >> 172.21.15.129:0/3184368087 conn(0x7f71200fc7d0 msgr2=0x7f7120105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 shutdown_connections 2024-09-05T01:33:26.272 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:26.269+0000 7f712774f640 1 -- 172.21.15.129:0/3184368087 wait complete. 2024-09-05T01:33:26.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:33:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:33:26.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:26 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:33:26.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T01:33:26.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T01:33:26.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:26 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' 2024-09-05T01:33:26.992 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T01:33:27.245 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:27 smithi129 ceph-mon[30979]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:27.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:27 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3184368087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:27 smithi194 ceph-mon[34941]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:27.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:27 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3184368087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:27.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:27 smithi183 ceph-mon[34589]: pgmap v897: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-05T01:33:27.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:27 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3184368087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T01:33:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3352569116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84106f20 msgr2=0x7f9e8410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3352569116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84106f20 0x7f9e8410eaa0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009ca0 tx=0x7f9e7c02a8c0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3352569116 shutdown_connections 2024-09-05T01:33:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3352569116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84106f20 0x7f9e8410eaa0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.912 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3352569116 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e841067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3352569116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84101df0 0x7f9e841021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3352569116 >> 172.21.15.129:0/3352569116 conn(0x7f9e840fd8b0 msgr2=0x7f9e840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3352569116 shutdown_connections 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.891+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3352569116 wait complete. 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 Processor -- start 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 -- start start 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e84074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 0x7f9e84074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84111f20 con 0x7f9e84101df0 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e84111da0 con 0x7f9e84106f20 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8bee2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f9e841120a0 con 0x7f9e841027a0 2024-09-05T01:33:27.913 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46710/0 (socket says 172.21.15.129:46710) 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 -- 172.21.15.129:0/3837683145 learned_addr learned my addr 172.21.15.129:0/3837683145 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8a47f640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 0x7f9e84074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 -- 172.21.15.129:0/3837683145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 msgr2=0x7f9e84074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8947d640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e84074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e84074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 -- 172.21.15.129:0/3837683145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 msgr2=0x7f9e84074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 0x7f9e84074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e89c7e640 1 -- 172.21.15.129:0/3837683145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e841a6840 con 0x7f9e84101df0 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8947d640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.892+0000 7f9e8a47f640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 0x7f9e84074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e89c7e640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000b4a0 tx=0x7f9e8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:27.914 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80014070 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e841a6b30 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e80010ac0 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e80017c30 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.893+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e841a7070 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.894+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e841021d0 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.895+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f9e80017070 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.896+0000 7f9e7affd640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 0x7f9e700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.896+0000 7f9e8947d640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 0x7f9e700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.896+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7f9e800cec30 con 0x7f9e84101df0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.896+0000 7f9e8947d640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 0x7f9e700a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9e74006fd0 tx=0x7f9e74008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:27.915 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:27.899+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e8009b720 con 0x7f9e84101df0 2024-09-05T01:33:28.069 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.068+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e8402d7c0 con 0x7f9e84101df0 2024-09-05T01:33:28.071 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.070+0000 7f9e7affd640 1 -- 172.21.15.129:0/3837683145 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+25009 (secure 0 0 0) 0x7f9e800bb0e0 con 0x7f9e84101df0 2024-09-05T01:33:28.071 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-05T01:33:28.072 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":93,"fsid":"c8062bdc-6b22-11ef-bcd6-c7b262605968","created":"2024-09-05T01:05:32.739177+0000","modified":"2024-09-05T01:26:34.410093+0000","last_up_change":"2024-09-05T01:12:14.581843+0000","last_in_change":"2024-09-05T01:11:50.922701+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T01:08:27.962073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T01:13:02.228925+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T01:13:04.722842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T01:13:11.133644+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a8916e3b-e0c4-45b9-84bb-5f527d0f0439","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6803","nonce":2948481734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6805","nonce":2948481734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6809","nonce":2948481734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2948481734},{"type":"v1","addr":"172.21.15.129:6807","nonce":2948481734}]},"public_addr":"172.21.15.129:6803/2948481734","cluster_addr":"172.21.15.129:6805/2948481734","heartbeat_back_addr":"172.21.15.129:6809/2948481734","heartbeat_front_addr":"172.21.15.129:6807/2948481734","state":["exists","up"]},{"osd":1,"uuid":"b6b99a10-04d7-40ff-8801-2acf7628a478","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6801","nonce":63466833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6803","nonce":63466833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6807","nonce":63466833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":63466833},{"type":"v1","addr":"172.21.15.183:6805","nonce":63466833}]},"public_addr":"172.21.15.183:6801/63466833","cluster_addr":"172.21.15.183:6803/63466833","heartbeat_back_addr":"172.21.15.183:6807/63466833","heartbeat_front_addr":"172.21.15.183:6805/63466833","state":["exists","up"]},{"osd":2,"uuid":"45de925c-c48d-4388-8e8d-3afcfd7c2022","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6800","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6801","nonce":968743504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6802","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6803","nonce":968743504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6806","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6807","nonce":968743504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6804","nonce":968743504},{"type":"v1","addr":"172.21.15.194:6805","nonce":968743504}]},"public_addr":"172.21.15.194:6801/968743504","cluster_addr":"172.21.15.194:6803/968743504","heartbeat_back_addr":"172.21.15.194:6807/968743504","heartbeat_front_addr":"172.21.15.194:6805/968743504","state":["exists","up"]},{"osd":3,"uuid":"9db73173-f27a-493d-a9c8-478d042973f5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6811","nonce":171450732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6813","nonce":171450732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6817","nonce":171450732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":171450732},{"type":"v1","addr":"172.21.15.129:6815","nonce":171450732}]},"public_addr":"172.21.15.129:6811/171450732","cluster_addr":"172.21.15.129:6813/171450732","heartbeat_back_addr":"172.21.15.129:6817/171450732","heartbeat_front_addr":"172.21.15.129:6815/171450732","state":["exists","up"]},{"osd":4,"uuid":"d31dce58-e63a-47ed-8940-9db4c09d6d1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6809","nonce":3612040515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6811","nonce":3612040515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6815","nonce":3612040515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3612040515},{"type":"v1","addr":"172.21.15.183:6813","nonce":3612040515}]},"public_addr":"172.21.15.183:6809/3612040515","cluster_addr":"172.21.15.183:6811/3612040515","heartbeat_back_addr":"172.21.15.183:6815/3612040515","heartbeat_front_addr":"172.21.15.183:6813/3612040515","state":["exists","up"]},{"osd":5,"uuid":"3c954db0-fb4f-4beb-99fe-abedf9d7e241","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6808","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6809","nonce":1401637167}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6810","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6811","nonce":1401637167}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6814","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6815","nonce":1401637167}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6812","nonce":1401637167},{"type":"v1","addr":"172.21.15.194:6813","nonce":1401637167}]},"public_addr":"172.21.15.194:6809/1401637167","cluster_addr":"172.21.15.194:6811/1401637167","heartbeat_back_addr":"172.21.15.194:6815/1401637167","heartbeat_front_addr":"172.21.15.194:6813/1401637167","state":["exists","up"]},{"osd":6,"uuid":"0213e2f7-1e56-4c20-9122-cd81c2dc4996","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6819","nonce":2103992741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6821","nonce":2103992741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6825","nonce":2103992741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2103992741},{"type":"v1","addr":"172.21.15.129:6823","nonce":2103992741}]},"public_addr":"172.21.15.129:6819/2103992741","cluster_addr":"172.21.15.129:6821/2103992741","heartbeat_back_addr":"172.21.15.129:6825/2103992741","heartbeat_front_addr":"172.21.15.129:6823/2103992741","state":["exists","up"]},{"osd":7,"uuid":"fd9aefa5-14e5-492a-8eb0-a0704322665a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6817","nonce":1669309278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6819","nonce":1669309278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6823","nonce":1669309278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":1669309278},{"type":"v1","addr":"172.21.15.183:6821","nonce":1669309278}]},"public_addr":"172.21.15.183:6817/1669309278","cluster_addr":"172.21.15.183:6819/1669309278","heartbeat_back_addr":"172.21.15.183:6823/1669309278","heartbeat_front_addr":"172.21.15.183:6821/1669309278","state":["exists","up"]},{"osd":8,"uuid":"f64d8289-4220-4110-b3f0-b17294db4107","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6816","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6817","nonce":307349616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6818","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6819","nonce":307349616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6822","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6823","nonce":307349616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6820","nonce":307349616},{"type":"v1","addr":"172.21.15.194:6821","nonce":307349616}]},"public_addr":"172.21.15.194:6817/307349616","cluster_addr":"172.21.15.194:6819/307349616","heartbeat_back_addr":"172.21.15.194:6823/307349616","heartbeat_front_addr":"172.21.15.194:6821/307349616","state":["exists","up"]},{"osd":9,"uuid":"5d297ed4-17e8-44e6-a4a2-70390bf762ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6827","nonce":1276233090}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6829","nonce":1276233090}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6833","nonce":1276233090}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":1276233090},{"type":"v1","addr":"172.21.15.129:6831","nonce":1276233090}]},"public_addr":"172.21.15.129:6827/1276233090","cluster_addr":"172.21.15.129:6829/1276233090","heartbeat_back_addr":"172.21.15.129:6833/1276233090","heartbeat_front_addr":"172.21.15.129:6831/1276233090","state":["exists","up"]},{"osd":10,"uuid":"2d90543e-16fb-4460-ad34-8e7d09c8c119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6825","nonce":1458641027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6827","nonce":1458641027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6831","nonce":1458641027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":1458641027},{"type":"v1","addr":"172.21.15.183:6829","nonce":1458641027}]},"public_addr":"172.21.15.183:6825/1458641027","cluster_addr":"172.21.15.183:6827/1458641027","heartbeat_back_addr":"172.21.15.183:6831/1458641027","heartbeat_front_addr":"172.21.15.183:6829/1458641027","state":["exists","up"]},{"osd":11,"uuid":"aa637f9f-a4b5-41aa-ab8f-66306b7c6565","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6824","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6825","nonce":3769265743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6826","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6827","nonce":3769265743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6830","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6831","nonce":3769265743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.194:6828","nonce":3769265743},{"type":"v1","addr":"172.21.15.194:6829","nonce":3769265743}]},"public_addr":"172.21.15.194:6825/3769265743","cluster_addr":"172.21.15.194:6827/3769265743","heartbeat_back_addr":"172.21.15.194:6831/3769265743","heartbeat_front_addr":"172.21.15.194:6829/3769265743","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:07:37.194540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:01.775926+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:25.183788+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:08:50.539685+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:14.607337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:09:39.897123+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:05.715901+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:30.605958+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:10:55.199375+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:21.232374+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:11:46.582379+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T01:12:12.308338+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":6}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1b","mappings":[{"from":5,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.6","mappings":[{"from":3,"to":7}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.129:6801/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3942407634":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/665628602":"2024-09-06T01:06:17.664286+0000","172.21.15.129:0/3304384691":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6800/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/2755277182":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4057892175":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/4214690776":"2024-09-06T01:06:17.664286+0000","172.21.15.129:6801/3490891928":"2024-09-06T01:05:54.362223+0000","172.21.15.129:0/3403026756":"2024-09-06T01:05:54.362223+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T01:33:28.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 msgr2=0x7f9e700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:28.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 0x7f9e700a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9e74006fd0 tx=0x7f9e74008040 comp rx=0 tx=0).stop 2024-09-05T01:33:28.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 msgr2=0x7f9e840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:28.073 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9e8000b4a0 tx=0x7f9e8000b960 comp rx=0 tx=0).stop 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 shutdown_connections 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f9e700a2b80 0x7f9e700a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e84106f20 0x7f9e84074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f9e841027a0 0x7f9e84074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 --2- 172.21.15.129:0/3837683145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e84101df0 0x7f9e840744f0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.072+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 >> 172.21.15.129:0/3837683145 conn(0x7f9e840fd8b0 msgr2=0x7f9e840feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.073+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 shutdown_connections 2024-09-05T01:33:28.074 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:28.073+0000 7f9e8bee2640 1 -- 172.21.15.129:0/3837683145 wait complete. 2024-09-05T01:33:28.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:28 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3837683145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:33:28.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:28 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3837683145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:33:28.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:28 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3837683145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T01:33:28.731 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs fail cephfs 2024-09-05T01:33:28.981 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:29.389 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:29 smithi129 ceph-mon[30979]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T01:33:29.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- 172.21.15.129:0/948668270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae20102da0 msgr2=0x7fae20103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:29.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 --2- 172.21.15.129:0/948668270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae20102da0 0x7fae20103180 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fae08009900 tx=0x7fae0802a480 comp rx=0 tx=0).stop 2024-09-05T01:33:29.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- 172.21.15.129:0/948668270 shutdown_connections 2024-09-05T01:33:29.492 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 --2- 172.21.15.129:0/948668270 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae2010c180 0x7fae2010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 --2- 172.21.15.129:0/948668270 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 0x7fae2010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 --2- 172.21.15.129:0/948668270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae20102da0 0x7fae20103180 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- 172.21.15.129:0/948668270 >> 172.21.15.129:0/948668270 conn(0x7fae200fc7a0 msgr2=0x7fae200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- 172.21.15.129:0/948668270 shutdown_connections 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- 172.21.15.129:0/948668270 wait complete. 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 Processor -- start 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.469+0000 7fae26eb2640 1 -- start start 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae20102da0 0x7fae2019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 0x7fae2019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:29.493 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111e50 con 0x7fae2010c180 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111cd0 con 0x7fae20102da0 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae26eb2640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fae20111fd0 con 0x7fae201036c0 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46730/0 (socket says 172.21.15.129:46730) 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 -- 172.21.15.129:0/19556874 learned_addr learned my addr 172.21.15.129:0/19556874 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae24c4e640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae20102da0 0x7fae2019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae1ffff640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 0x7fae2019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 -- 172.21.15.129:0/19556874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 msgr2=0x7fae2019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 0x7fae2019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.494 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 -- 172.21.15.129:0/19556874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae20102da0 msgr2=0x7fae2019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae20102da0 0x7fae2019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 -- 172.21.15.129:0/19556874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019f5e0 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.470+0000 7fae2544f640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fae18009fd0 tx=0x7fae1800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.471+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18011830 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.471+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019f8d0 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.471+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae18016070 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.471+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18010de0 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.471+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae201a71b0 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.472+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae20103180 con 0x7fae2010c180 2024-09-05T01:33:29.495 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.473+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fae180119d0 con 0x7fae2010c180 2024-09-05T01:33:29.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.474+0000 7fae1dffb640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 0x7fae0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:29.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.474+0000 7fae24c4e640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 0x7fae0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:29.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.474+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9266+0+0 (secure 0 0 0) 0x7fae180ceb60 con 0x7fae2010c180 2024-09-05T01:33:29.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.474+0000 7fae24c4e640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 0x7fae0c0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fae08031440 tx=0x7fae08005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:29.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.478+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae1809b6b0 con 0x7fae2010c180 2024-09-05T01:33:29.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-mon[34941]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T01:33:29.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-mon[34589]: pgmap v898: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.4 KiB/s wr, 8 op/s 2024-09-05T01:33:29.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.678+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fae201a74e0 con 0x7fae2010c180 2024-09-05T01:33:29.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.680+0000 7fae1dffb640 1 -- 172.21.15.129:0/19556874 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7fae1809afb0 con 0x7fae2010c180 2024-09-05T01:33:29.681 INFO:teuthology.orchestra.run.smithi129.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 msgr2=0x7fae0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 0x7fae0c0a5040 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fae08031440 tx=0x7fae08005e80 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 msgr2=0x7fae2019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fae18009fd0 tx=0x7fae1800b8e0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 shutdown_connections 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fae0c0a2b80 0x7fae0c0a5040 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae2010c180 0x7fae2019ef00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fae201036c0 0x7fae2019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 --2- 172.21.15.129:0/19556874 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae20102da0 0x7fae2019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:29.685 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 >> 172.21.15.129:0/19556874 conn(0x7fae200fc7a0 msgr2=0x7fae20101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 shutdown_connections 2024-09-05T01:33:29.686 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:29.684+0000 7fae26eb2640 1 -- 172.21.15.129:0/19556874 wait complete. 2024-09-05T01:33:29.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:29 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a[30975]: 2024-09-05T01:33:29.679+0000 7f3a8a372640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T01:33:30.000 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-mds[65080]: -- 172.21.15.194:0/3643276328 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56517f3d7c20 con 0x56517f3b5400 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: starting mds.f at 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-mds[65377]: -- 172.21.15.194:0/3075549506 <== mon.1 v2:172.21.15.194:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d50ccb7860 con 0x55d50cc95000 2024-09-05T01:33:30.001 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: starting mds.c at 2024-09-05T01:33:30.002 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i[65672]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.002 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i[65672]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.002 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-mds[65676]: -- 172.21.15.194:0/3475072631 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620d1c5dc20 con 0x5620d1c3b400 2024-09-05T01:33:30.002 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:33:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i[65672]: starting mds.i at 2024-09-05T01:33:30.161 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.161 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-mds[64616]: -- 172.21.15.183:0/361780379 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629b0ff7860 con 0x5629b0fd5000 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: starting mds.b at 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-mds[64315]: -- 172.21.15.183:0/3572797084 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56243ae45860 con 0x56243ae23000 2024-09-05T01:33:30.162 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: starting mds.h at 2024-09-05T01:33:30.163 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: ignoring --setuser ceph since I am not root 2024-09-05T01:33:30.163 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: ignoring --setgroup ceph since I am not root 2024-09-05T01:33:30.163 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-mds[64012]: -- 172.21.15.183:0/1382685351 <== mon.0 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562222ad1c20 con 0x562222aae800 2024-09-05T01:33:30.163 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:33:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: starting mds.e at 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T01:33:30.339 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:30 smithi129 ceph-mon[30979]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-05T01:33:30.368 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-05T01:33:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T01:33:30.500 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T01:33:30.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T01:33:30.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T01:33:30.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T01:33:30.501 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:30 smithi194 ceph-mon[34941]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-05T01:33:30.618 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/19556874' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T01:33:30.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:30 smithi183 ceph-mon[34589]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 -- 172.21.15.129:0/2080536681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 msgr2=0x7fdf0c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/2080536681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c10bb50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fdef4009900 tx=0x7fdef402a480 comp rx=0 tx=0).stop 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 -- 172.21.15.129:0/2080536681 shutdown_connections 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/2080536681 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c10c170 0x7fdf0c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/2080536681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c10bb50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/2080536681 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c102d40 0x7fdf0c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 -- 172.21.15.129:0/2080536681 >> 172.21.15.129:0/2080536681 conn(0x7fdf0c0fc7a0 msgr2=0x7fdf0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 -- 172.21.15.129:0/2080536681 shutdown_connections 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.111+0000 7fdf10bcd640 1 -- 172.21.15.129:0/2080536681 wait complete. 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 Processor -- start 2024-09-05T01:33:31.125 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 -- start start 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c102d40 0x7fdf0c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 0x7fdf0c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c111ee0 con 0x7fdf0c103660 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c111d60 con 0x7fdf0c10c170 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf10bcd640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fdf0c112060 con 0x7fdf0c102d40 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf03fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:46738/0 (socket says 172.21.15.129:46738) 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf03fff640 1 -- 172.21.15.129:0/3451254383 learned_addr learned my addr 172.21.15.129:0/3451254383 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf0a59c640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c102d40 0x7fdf0c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:31.126 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.112+0000 7fdf0ad9d640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 0x7fdf0c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 -- 172.21.15.129:0/3451254383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c102d40 msgr2=0x7fdf0c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c102d40 0x7fdf0c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 -- 172.21.15.129:0/3451254383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 msgr2=0x7fdf0c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 0x7fdf0c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 -- 172.21.15.129:0/3451254383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf0c19d4c0 con 0x7fdf0c103660 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf0ad9d640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 0x7fdf0c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf03fff640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdef4006010 tx=0x7fdef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef403f030 con 0x7fdf0c103660 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0c19d750 con 0x7fdf0c103660 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdef4003800 con 0x7fdf0c103660 2024-09-05T01:33:31.127 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef4035070 con 0x7fdf0c103660 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.113+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf0c1a5090 con 0x7fdf0c103660 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.114+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf0c1031a0 con 0x7fdf0c103660 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.115+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fdef40039b0 con 0x7fdf0c103660 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.116+0000 7fdf037fe640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 0x7fdef80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.116+0000 7fdf0a59c640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 0x7fdef80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.116+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9524+0+0 (secure 0 0 0) 0x7fdef40efff0 con 0x7fdf0c103660 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.116+0000 7fdf0a59c640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 0x7fdef80a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:31.128 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.119+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdef40bca40 con 0x7fdf0c103660 2024-09-05T01:33:31.322 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.321+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdf0c107210 con 0x7fdf0c103660 2024-09-05T01:33:31.694 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.692+0000 7fdf037fe640 1 -- 172.21.15.129:0/3451254383 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) ==== 104+0+0 (secure 0 0 0) 0x7fdef40bc340 con 0x7fdf0c103660 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 msgr2=0x7fdef80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 0x7fdef80a5040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009990 tx=0x7fdefc009110 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 msgr2=0x7fdf0c198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdef4006010 tx=0x7fdef40042f0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 shutdown_connections 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fdef80a2b80 0x7fdef80a5040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdf0c10c170 0x7fdf0c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdf0c103660 0x7fdf0c198a50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 --2- 172.21.15.129:0/3451254383 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fdf0c102d40 0x7fdf0c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 >> 172.21.15.129:0/3451254383 conn(0x7fdf0c0fc7a0 msgr2=0x7fdf0c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:31.697 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 shutdown_connections 2024-09-05T01:33:31.698 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:31.695+0000 7fdf10bcd640 1 -- 172.21.15.129:0/3451254383 wait complete. 2024-09-05T01:33:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-05T01:33:31.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6834/1569984174,v1:172.21.15.194:6835/1569984174] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6834/871709512,v1:172.21.15.183:6835/871709512] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6836/3442578811,v1:172.21.15.183:6838/3442578811] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.183:6837/1607289754,v1:172.21.15.183:6839/1607289754] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6838/708449970,v1:172.21.15.194:6839/708449970] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: mds.? [v2:172.21.15.194:6836/3826956450,v1:172.21.15.194:6837/3826956450] up:boot 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:33:31.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:33:31.994 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:31 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:33:32.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6834/1569984174,v1:172.21.15.194:6835/1569984174] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6834/871709512,v1:172.21.15.183:6835/871709512] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6836/3442578811,v1:172.21.15.183:6838/3442578811] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.183:6837/1607289754,v1:172.21.15.183:6839/1607289754] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6838/708449970,v1:172.21.15.194:6839/708449970] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: mds.? [v2:172.21.15.194:6836/3826956450,v1:172.21.15.194:6837/3826956450] up:boot 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:33:32.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:33:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:33:32.002 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:31 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: pgmap v900: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 921 B/s wr, 6 op/s 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6834/1569984174,v1:172.21.15.194:6835/1569984174] up:boot 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6834/871709512,v1:172.21.15.183:6835/871709512] up:boot 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6836/3442578811,v1:172.21.15.183:6838/3442578811] up:boot 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.183:6837/1607289754,v1:172.21.15.183:6839/1607289754] up:boot 2024-09-05T01:33:32.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6838/708449970,v1:172.21.15.194:6839/708449970] up:boot 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: mds.? [v2:172.21.15.194:6836/3826956450,v1:172.21.15.194:6837/3826956450] up:boot 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='mgr.14150 172.21.15.129:0/2366036160' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T01:33:32.162 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:31 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T01:33:32.390 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-05T01:33:32.641 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:32 smithi129 ceph-mon[30979]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T01:33:32.992 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:32 smithi129 ceph-mon[30979]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T01:33:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:32 smithi129 ceph-mon[30979]: Cluster is now healthy 2024-09-05T01:33:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:32 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:33:32.993 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:32 smithi129 ceph-mon[30979]: fsmap 12 up:standby 2024-09-05T01:33:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:32 smithi194 ceph-mon[34941]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T01:33:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:32 smithi194 ceph-mon[34941]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T01:33:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:32 smithi194 ceph-mon[34941]: Cluster is now healthy 2024-09-05T01:33:33.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:32 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:33:33.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:32 smithi194 ceph-mon[34941]: fsmap 12 up:standby 2024-09-05T01:33:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:32 smithi183 ceph-mon[34589]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T01:33:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:32 smithi183 ceph-mon[34589]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T01:33:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:32 smithi183 ceph-mon[34589]: Cluster is now healthy 2024-09-05T01:33:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:32 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3451254383' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T01:33:33.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:32 smithi183 ceph-mon[34589]: fsmap 12 up:standby 2024-09-05T01:33:33.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- 172.21.15.129:0/3453409063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f0103660 msgr2=0x7fe0f010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:33.283 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/3453409063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f0103660 0x7fe0f010bb50 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4009900 tx=0x7fe0e402a480 comp rx=0 tx=0).stop 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- 172.21.15.129:0/3453409063 shutdown_connections 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/3453409063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f010c170 0x7fe0f010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/3453409063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f0103660 0x7fe0f010bb50 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/3453409063 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 0x7fe0f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- 172.21.15.129:0/3453409063 >> 172.21.15.129:0/3453409063 conn(0x7fe0f00fc7a0 msgr2=0x7fe0f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- 172.21.15.129:0/3453409063 shutdown_connections 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- 172.21.15.129:0/3453409063 wait complete. 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 Processor -- start 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 -- start start 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 0x7fe0f01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:33.284 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 0x7fe0f0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.273+0000 7fe0f6d43640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f6d43640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0111e80 con 0x7fe0f010c170 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f6d43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0111d00 con 0x7fe0f0103660 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f6d43640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f0112000 con 0x7fe0f0102d40 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42892/0 (socket says 172.21.15.129:42892) 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 -- 172.21.15.129:0/1622318691 learned_addr learned my addr 172.21.15.129:0/1622318691 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f4adf640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 0x7fe0f01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0effff640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 0x7fe0f0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 -- 172.21.15.129:0/1622318691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 msgr2=0x7fe0f01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:33.285 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 0x7fe0f01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 -- 172.21.15.129:0/1622318691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 msgr2=0x7fe0f0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 0x7fe0f0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 -- 172.21.15.129:0/1622318691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01a68d0 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0effff640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 0x7fe0f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f52e0640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b970 tx=0x7fe0e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8019070 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe0e8014440 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80075d0 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.274+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f01a6bc0 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.275+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01a7080 con 0x7fe0f010c170 2024-09-05T01:33:33.286 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.275+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0103120 con 0x7fe0f010c170 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.276+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fe0e801c070 con 0x7fe0f010c170 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.277+0000 7fe0edffb640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 0x7fe0d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.277+0000 7fe0f4adf640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 0x7fe0d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.277+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9524+0+0 (secure 0 0 0) 0x7fe0e80ceb20 con 0x7fe0f010c170 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.278+0000 7fe0f4adf640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 0x7fe0d80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005fd0 tx=0x7fe0e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:33.287 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.282+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0e809af20 con 0x7fe0f010c170 2024-09-05T01:33:33.452 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:33.450+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe0f01071e0 con 0x7fe0f010c170 2024-09-05T01:33:33.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:33 smithi129 ceph-mon[30979]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 819 B/s wr, 6 op/s 2024-09-05T01:33:33.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:33 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:34.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:33 smithi194 ceph-mon[34941]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 819 B/s wr, 6 op/s 2024-09-05T01:33:34.001 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:33 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:34.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:33 smithi183 ceph-mon[34589]: pgmap v901: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 819 B/s wr, 6 op/s 2024-09-05T01:33:34.161 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:33 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:34.347 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.345+0000 7fe0edffb640 1 -- 172.21.15.129:0/1622318691 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7fe0e809f220 con 0x7fe0f010c170 2024-09-05T01:33:34.347 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_metadata' removed 2024-09-05T01:33:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.348+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 msgr2=0x7fe0d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.348+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 0x7fe0d80a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0005fd0 tx=0x7fe0e000b390 comp rx=0 tx=0).stop 2024-09-05T01:33:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.348+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 msgr2=0x7fe0f0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:34.350 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.348+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b970 tx=0x7fe0e800be30 comp rx=0 tx=0).stop 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 shutdown_connections 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fe0d80a2b80 0x7fe0d80a5040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0f010c170 0x7fe0f0101420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0f0103660 0x7fe0f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 --2- 172.21.15.129:0/1622318691 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fe0f0102d40 0x7fe0f01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:34.351 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 >> 172.21.15.129:0/1622318691 conn(0x7fe0f00fc7a0 msgr2=0x7fe0f00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 shutdown_connections 2024-09-05T01:33:34.352 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:34.349+0000 7fe0f6d43640 1 -- 172.21.15.129:0/1622318691 wait complete. 2024-09-05T01:33:35.058 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-05T01:33:35.310 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:35.634 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:35 smithi129 ceph-mon[30979]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-09-05T01:33:35.634 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:35 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:35.634 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:35 smithi129 ceph-mon[30979]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T01:33:35.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:35 smithi183 ceph-mon[34589]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-09-05T01:33:35.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:35 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:35.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:35 smithi183 ceph-mon[34589]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T01:33:35.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:35 smithi194 ceph-mon[34941]: pgmap v902: 129 pgs: 129 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 818 B/s wr, 4 op/s 2024-09-05T01:33:35.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:35 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1622318691' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:35.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:35 smithi194 ceph-mon[34941]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T01:33:35.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 -- 172.21.15.129:0/430958623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 msgr2=0x7fa768106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:35.784 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 --2- 172.21.15.129:0/430958623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa768106e40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa758009930 tx=0x7fa75802a420 comp rx=0 tx=0).stop 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 -- 172.21.15.129:0/430958623 shutdown_connections 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 --2- 172.21.15.129:0/430958623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 0x7fa76810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 --2- 172.21.15.129:0/430958623 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa768100a60 0x7fa768100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 --2- 172.21.15.129:0/430958623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa768106e40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 -- 172.21.15.129:0/430958623 >> 172.21.15.129:0/430958623 conn(0x7fa7680fc780 msgr2=0x7fa7680feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 -- 172.21.15.129:0/430958623 shutdown_connections 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.767+0000 7fa76d447640 1 -- 172.21.15.129:0/430958623 wait complete. 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 Processor -- start 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 -- start start 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa768100a60 0x7fa768199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:35.785 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 0x7fa768199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa768110d30 con 0x7fa768106a60 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa768110bb0 con 0x7fa768101400 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa76d447640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7fa768110eb0 con 0x7fa768100a60 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa7677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa7677fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42908/0 (socket says 172.21.15.129:42908) 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa7677fe640 1 -- 172.21.15.129:0/1349885975 learned_addr learned my addr 172.21.15.129:0/1349885975 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.768+0000 7fa75e7fc640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 0x7fa768199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 -- 172.21.15.129:0/1349885975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa768100a60 msgr2=0x7fa768199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa768100a60 0x7fa768199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 -- 172.21.15.129:0/1349885975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 msgr2=0x7fa768199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 0x7fa768199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:35.786 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 -- 172.21.15.129:0/1349885975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7681a47c0 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa7677fe640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa76000cce0 tx=0x7fa76000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76000f450 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa76000faa0 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7681a4990 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa760018a80 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.769+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7681a4e50 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.770+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa768100ec0 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.771+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7fa76000fc50 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.771+0000 7fa764ff9640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 0x7fa7540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.772+0000 7fa766ffd640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 0x7fa7540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.772+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8912+0+0 (secure 0 0 0) 0x7fa7600cf7b0 con 0x7fa768106a60 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.772+0000 7fa766ffd640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 0x7fa7540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7580096c0 tx=0x7fa758031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:35.787 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.776+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa760014030 con 0x7fa768106a60 2024-09-05T01:33:35.945 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:35.943+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa76802d7c0 con 0x7fa768106a60 2024-09-05T01:33:36.358 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.355+0000 7fa764ff9640 1 -- 172.21.15.129:0/1349885975 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fa76009c470 con 0x7fa768106a60 2024-09-05T01:33:36.358 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data' removed 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.357+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 msgr2=0x7fa7540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.357+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 0x7fa7540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7580096c0 tx=0x7fa758031400 comp rx=0 tx=0).stop 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.357+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 msgr2=0x7fa7681a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.357+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fa76000cce0 tx=0x7fa76000a620 comp rx=0 tx=0).stop 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 shutdown_connections 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7fa7540a2b80 0x7fa7540a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa768106a60 0x7fa7681a4280 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa768101400 0x7fa768199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:36.360 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 --2- 172.21.15.129:0/1349885975 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7fa768100a60 0x7fa768199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:36.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 >> 172.21.15.129:0/1349885975 conn(0x7fa7680fc780 msgr2=0x7fa7680fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:36.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 shutdown_connections 2024-09-05T01:33:36.361 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:36.358+0000 7fa76d447640 1 -- 172.21.15.129:0/1349885975 wait complete. 2024-09-05T01:33:36.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:36 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:36.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:36 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:36.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:36 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:37.085 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4 shell --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-05T01:33:37.333 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T01:33:37.612 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:37 smithi129 ceph-mon[30979]: pgmap v904: 97 pgs: 97 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:37.612 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:37 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:37.612 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:37 smithi129 ceph-mon[30979]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T01:33:37.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:37 smithi183 ceph-mon[34589]: pgmap v904: 97 pgs: 97 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:37.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:37 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:37.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:37 smithi183 ceph-mon[34589]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T01:33:37.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:37 smithi194 ceph-mon[34941]: pgmap v904: 97 pgs: 97 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:37.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:37 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/1349885975' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:37.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:37 smithi194 ceph-mon[34941]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T01:33:37.810 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 -- 172.21.15.129:0/3668632793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 msgr2=0x7f321810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 --2- 172.21.15.129:0/3668632793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f321810bb50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3200009900 tx=0x7f320002a480 comp rx=0 tx=0).stop 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 -- 172.21.15.129:0/3668632793 shutdown_connections 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 --2- 172.21.15.129:0/3668632793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 0x7f321810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 --2- 172.21.15.129:0/3668632793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f321810bb50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 --2- 172.21.15.129:0/3668632793 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 -- 172.21.15.129:0/3668632793 >> 172.21.15.129:0/3668632793 conn(0x7f32180fc7a0 msgr2=0x7f32180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 -- 172.21.15.129:0/3668632793 shutdown_connections 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 -- 172.21.15.129:0/3668632793 wait complete. 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.792+0000 7f321e625640 1 Processor -- start 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 -- start start 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 --2- >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:37.811 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 0x7f321819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3218111e20 con 0x7f3218103660 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3218111ca0 con 0x7f321810c170 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321e625640 1 -- --> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] -- mon_getmap magic: 0 -- 0x7f3218111fa0 con 0x7f3218102d40 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:42920/0 (socket says 172.21.15.129:42920) 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 -- 172.21.15.129:0/3179015583 learned_addr learned my addr 172.21.15.129:0/3179015583 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f321cbc2640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 0x7f321819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f3217fff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 -- 172.21.15.129:0/3179015583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 msgr2=0x7f3218198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 -- 172.21.15.129:0/3179015583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 msgr2=0x7f321819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:37.812 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 0x7f321819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 -- 172.21.15.129:0/3179015583 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321819d320 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f3217fff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f320ffff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3200005940 tx=0x7f32000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.793+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320003f030 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.794+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3200003cb0 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.794+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321819d5b0 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.794+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200035070 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.794+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32181a4ef0 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.795+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f321806b070 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.796+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 20) ==== 137834+0+0 (secure 0 0 0) 0x7f3200042030 con 0x7f3218103660 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.796+0000 7f3215ffb640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 0x7f32040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.796+0000 7f3217fff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 0x7f32040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T01:33:37.813 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.796+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8324+0+0 (secure 0 0 0) 0x7f32000ef920 con 0x7f3218103660 2024-09-05T01:33:37.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.797+0000 7f3217fff640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 0x7f32040a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3208009b40 tx=0x7f3208005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T01:33:37.814 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.800+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32000bc7c0 con 0x7f3218103660 2024-09-05T01:33:37.968 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:37.967+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32181071e0 con 0x7f3218103660 2024-09-05T01:33:38.367 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.365+0000 7f3215ffb640 1 -- 172.21.15.129:0/3179015583 <== mon.0 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v97) ==== 178+0+0 (secure 0 0 0) 0x7f32000bc0c0 con 0x7f3218103660 2024-09-05T01:33:38.367 INFO:teuthology.orchestra.run.smithi129.stderr:pool 'cephfs_data_ec' removed 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.368+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 msgr2=0x7f32040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.368+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 0x7f32040a5040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3208009b40 tx=0x7f3208005e50 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 msgr2=0x7f32181988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3200005940 tx=0x7f32000042f0 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 shutdown_connections 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:6800/655841650,v1:172.21.15.129:6801/655841650] conn(0x7f32040a2b80 0x7f32040a5040 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3208009b40 tx=0x7f3208005e50 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f321810c170 0x7f321819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3218103660 0x7f32181988b0 secure :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3200005940 tx=0x7f32000042f0 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 --2- 172.21.15.129:0/3179015583 >> [v2:172.21.15.194:3300/0,v1:172.21.15.194:6789/0] conn(0x7f3218102d40 0x7f3218198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T01:33:38.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.369+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 >> 172.21.15.129:0/3179015583 conn(0x7f32180fc7a0 msgr2=0x7f32180fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T01:33:38.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.375+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 shutdown_connections 2024-09-05T01:33:38.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-05T01:33:38.376+0000 7f321e625640 1 -- 172.21.15.129:0/3179015583 wait complete. 2024-09-05T01:33:38.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:38 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:38.742 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:38 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:38.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:38 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T01:33:39.277 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:33:39.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:33:39.432 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:33:39.471 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-05T01:33:39.472 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T01:33:39.513 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T01:33:39.538 DEBUG:teuthology.orchestra.run.smithi194:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T01:33:39.562 INFO:tasks.cephadm:Stopping all daemons... 2024-09-05T01:33:39.562 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-05T01:33:39.562 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a 2024-09-05T01:33:39.585 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 ceph-mon[30979]: pgmap v906: 33 pgs: 33 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:39.585 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 ceph-mon[30979]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:39.586 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 ceph-mon[30979]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T01:33:39.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:39 smithi183 ceph-mon[34589]: pgmap v906: 33 pgs: 33 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:39.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:39 smithi183 ceph-mon[34589]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:39.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:39 smithi183 ceph-mon[34589]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T01:33:39.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:39 smithi194 ceph-mon[34941]: pgmap v906: 33 pgs: 33 active+clean; 1.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T01:33:39.750 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:39 smithi194 ceph-mon[34941]: from='client.? 172.21.15.129:0/3179015583' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T01:33:39.751 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:39 smithi194 ceph-mon[34941]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T01:33:39.904 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 systemd[1]: Stopping Ceph mon.a for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:40.178 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a[30975]: 2024-09-05T01:33:39.902+0000 7f3a8fb7d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:40.178 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a[30975]: 2024-09-05T01:33:39.902+0000 7f3a8fb7d640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-05T01:33:40.492 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:40 smithi129 podman[108221]: 2024-09-05 01:33:40.178387378 +0000 UTC m=+0.388388668 container died ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-05T01:33:40.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:40 smithi129 podman[108221]: 2024-09-05 01:33:40.628699854 +0000 UTC m=+0.838701142 container cleanup ea8dbedddb59b947043a66d15c9243e38093e3a52958b64ec6be4cfe73fbf04e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T01:33:40.989 INFO:journalctl@ceph.mon.a.smithi129.stdout:Sep 05 01:33:40 smithi129 bash[108221]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-a 2024-09-05T01:33:41.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.a.service' 2024-09-05T01:33:41.201 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:41.201 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-05T01:33:41.201 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-05T01:33:41.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.b 2024-09-05T01:33:41.661 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:41 smithi183 systemd[1]: Stopping Ceph mon.b for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:42.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-b[34585]: 2024-09-05T01:33:41.672+0000 7f307e373640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:42.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-b[34585]: 2024-09-05T01:33:41.672+0000 7f307e373640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-05T01:33:42.041 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:41 smithi183 podman[69884]: 2024-09-05 01:33:41.744744855 +0000 UTC m=+0.205156767 container died 6b76040b418999c21763b07e1648583496c46e7c91d2819e49d781636e867055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-b, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-05T01:33:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:42 smithi183 podman[69884]: 2024-09-05 01:33:42.140166952 +0000 UTC m=+0.600578853 container cleanup 6b76040b418999c21763b07e1648583496c46e7c91d2819e49d781636e867055 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-05T01:33:42.411 INFO:journalctl@ceph.mon.b.smithi183.stdout:Sep 05 01:33:42 smithi183 bash[69884]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-b 2024-09-05T01:33:42.618 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.b.service' 2024-09-05T01:33:42.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:42.665 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-05T01:33:42.665 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-05T01:33:42.665 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.c 2024-09-05T01:33:43.000 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:42 smithi194 systemd[1]: Stopping Ceph mon.c for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:43.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-c[34925]: 2024-09-05T01:33:43.146+0000 7fbe2ff54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:43.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-c[34925]: 2024-09-05T01:33:43.146+0000 7fbe2ff54640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-05T01:33:43.277 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:43 smithi194 podman[71480]: 2024-09-05 01:33:43.167901611 +0000 UTC m=+0.152749001 container died 028c365edf14f75bcb5387a1ad8a45b318412de6bdc5fcc9f5109318bf64b464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-05T01:33:44.183 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:43 smithi194 podman[71480]: 2024-09-05 01:33:43.831692578 +0000 UTC m=+0.816539975 container cleanup 028c365edf14f75bcb5387a1ad8a45b318412de6bdc5fcc9f5109318bf64b464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-05T01:33:44.183 INFO:journalctl@ceph.mon.c.smithi194.stdout:Sep 05 01:33:43 smithi194 bash[71480]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mon-c 2024-09-05T01:33:44.317 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mon.c.service' 2024-09-05T01:33:44.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:44.365 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-05T01:33:44.365 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-05T01:33:44.365 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T01:33:44.710 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:44 smithi129 systemd[1]: Stopping Ceph mgr.x for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:44.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:33:44.708+0000 7f278f425640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:44.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x[31204]: 2024-09-05T01:33:44.708+0000 7f278f425640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-05T01:33:44.992 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:44 smithi129 podman[108336]: 2024-09-05 01:33:44.740348974 +0000 UTC m=+0.179464757 container died 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:33:45.404 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 podman[108336]: 2024-09-05 01:33:45.151591523 +0000 UTC m=+0.590707303 container cleanup 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T01:33:45.404 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 bash[108336]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x 2024-09-05T01:33:45.642 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service' 2024-09-05T01:33:45.664 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 podman[108348]: 2024-09-05 01:33:45.403801084 +0000 UTC m=+0.659626426 container remove 5ac36ba88b1dd75ca1c9faea2c98f71fd0626057f29b6e0012d610d1f7779db8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:91c8a7e56f7ba6f8ae8d6d8c0ae8cae96c5619e4, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-x, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:33:45.664 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-05T01:33:45.665 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 systemd[1]: Stopped Ceph mgr.x for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:33:45.665 INFO:journalctl@ceph.mgr.x.smithi129.stdout:Sep 05 01:33:45 smithi129 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.x.service: Consumed 48.158s CPU time. 2024-09-05T01:33:45.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:45.678 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-05T01:33:45.678 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-05T01:33:45.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y 2024-09-05T01:33:46.016 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:45 smithi183 systemd[1]: Stopping Ceph mgr.y for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:46.338 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 podman[70008]: 2024-09-05 01:33:46.015690942 +0000 UTC m=+0.145453118 container died a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-05T01:33:46.661 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 podman[70008]: 2024-09-05 01:33:46.437419026 +0000 UTC m=+0.567181198 container cleanup a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-05T01:33:46.661 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 bash[70008]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y 2024-09-05T01:33:46.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y.service' 2024-09-05T01:33:46.936 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 podman[70022]: 2024-09-05 01:33:46.67244159 +0000 UTC m=+0.648889627 container remove a09de669900d1dc98a3a634cfb2beed763aa55e4bec8dbcc4492ff9d273b040e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:33:46.937 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-05T01:33:46.937 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-09-05T01:33:46.937 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 systemd[1]: Stopped Ceph mgr.y for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:33:46.937 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Sep 05 01:33:46 smithi183 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.y.service: Consumed 6.859s CPU time. 2024-09-05T01:33:46.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:46.950 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-05T01:33:46.950 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-05T01:33:46.950 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.z 2024-09-05T01:33:47.250 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:33:47 smithi194 systemd[1]: Stopping Ceph mgr.z for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:47.855 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:33:47 smithi194 podman[71617]: 2024-09-05 01:33:47.524422378 +0000 UTC m=+0.371604968 container stop 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e) 2024-09-05T01:33:47.855 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:33:47 smithi194 podman[71617]: 2024-09-05 01:33:47.550217717 +0000 UTC m=+0.397400309 container died 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-05T01:33:48.189 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:33:47 smithi194 podman[71617]: 2024-09-05 01:33:47.954298722 +0000 UTC m=+0.801481312 container cleanup 4325e8e658f86965fffd6bf4a593e15eb25928c5feae2befc4ba41e0deec0eac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T01:33:48.189 INFO:journalctl@ceph.mgr.z.smithi194.stdout:Sep 05 01:33:47 smithi194 bash[71617]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mgr-z 2024-09-05T01:33:48.406 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mgr.z.service' 2024-09-05T01:33:48.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:48.441 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-05T01:33:48.441 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-05T01:33:48.441 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.0 2024-09-05T01:33:48.742 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:48 smithi129 systemd[1]: Stopping Ceph osd.0 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:49.492 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:33:49.065+0000 7f786a3d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:49.492 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:33:49.065+0000 7f786a3d5640 -1 osd.0 97 *** Got signal Terminated *** 2024-09-05T01:33:49.492 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0[44935]: 2024-09-05T01:33:49.065+0000 7f786a3d5640 -1 osd.0 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:33:54.899 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:54 smithi129 podman[108458]: 2024-09-05 01:33:54.626320662 +0000 UTC m=+5.986762896 container died 559f496a68432e5c58475abd749adfbc8ca7a2c7da5e33be9007d8781d2b24bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-05T01:33:55.242 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 podman[108458]: 2024-09-05 01:33:55.066117442 +0000 UTC m=+6.426559665 container cleanup 559f496a68432e5c58475abd749adfbc8ca7a2c7da5e33be9007d8781d2b24bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:33:55.243 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 bash[108458]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0 2024-09-05T01:33:55.723 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 podman[108473]: 2024-09-05 01:33:55.317881658 +0000 UTC m=+0.688304933 container remove 559f496a68432e5c58475abd749adfbc8ca7a2c7da5e33be9007d8781d2b24bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T01:33:55.992 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 podman[108564]: 2024-09-05 01:33:55.625827083 +0000 UTC m=+0.015817208 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:33:55.992 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 podman[108564]: 2024-09-05 01:33:55.744020377 +0000 UTC m=+0.134010485 container create a7ac6417bf114c7f5cafaae0df11f41d769f26a7ceba42edc375b4f2a80310c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-05T01:33:56.362 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:55 smithi129 podman[108564]: 2024-09-05 01:33:55.996729867 +0000 UTC m=+0.386719979 container init a7ac6417bf114c7f5cafaae0df11f41d769f26a7ceba42edc375b4f2a80310c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-05T01:33:56.362 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:56 smithi129 podman[108564]: 2024-09-05 01:33:56.000454357 +0000 UTC m=+0.390444486 container start a7ac6417bf114c7f5cafaae0df11f41d769f26a7ceba42edc375b4f2a80310c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:33:56.362 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:56 smithi129 podman[108564]: 2024-09-05 01:33:56.121953026 +0000 UTC m=+0.511943147 container attach a7ac6417bf114c7f5cafaae0df11f41d769f26a7ceba42edc375b4f2a80310c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:33:56.635 INFO:journalctl@ceph.osd.0.smithi129.stdout:Sep 05 01:33:56 smithi129 podman[108564]: 2024-09-05 01:33:56.361930221 +0000 UTC m=+0.751920336 container died a7ac6417bf114c7f5cafaae0df11f41d769f26a7ceba42edc375b4f2a80310c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux ) 2024-09-05T01:33:57.003 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.0.service' 2024-09-05T01:33:57.044 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:33:57.044 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-05T01:33:57.045 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-05T01:33:57.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.1 2024-09-05T01:33:57.367 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:33:57 smithi183 systemd[1]: Stopping Ceph osd.1 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:33:57.661 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:33:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:33:57.366+0000 7fa10fa92640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:33:57.661 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:33:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:33:57.366+0000 7fa10fa92640 -1 osd.1 97 *** Got signal Terminated *** 2024-09-05T01:33:57.661 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:33:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1[38901]: 2024-09-05T01:33:57.366+0000 7fa10fa92640 -1 osd.1 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:03.209 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:02 smithi183 podman[70132]: 2024-09-05 01:34:02.915193918 +0000 UTC m=+5.674524235 container died 0151a377b466a559ce33e4dbd827d6240900bc857b871d4b9fd79cf56c17b75d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-05T01:34:03.577 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:03 smithi183 podman[70132]: 2024-09-05 01:34:03.316612071 +0000 UTC m=+6.075942396 container cleanup 0151a377b466a559ce33e4dbd827d6240900bc857b871d4b9fd79cf56c17b75d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T01:34:03.577 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:03 smithi183 bash[70132]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1 2024-09-05T01:34:03.894 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:03 smithi183 podman[70147]: 2024-09-05 01:34:03.576574364 +0000 UTC m=+0.664228236 container remove 0151a377b466a559ce33e4dbd827d6240900bc857b871d4b9fd79cf56c17b75d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_CLEAN=True) 2024-09-05T01:34:04.161 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:03 smithi183 podman[70238]: 2024-09-05 01:34:03.797020036 +0000 UTC m=+0.016102454 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:04.161 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:03 smithi183 podman[70238]: 2024-09-05 01:34:03.919164994 +0000 UTC m=+0.138247394 container create a63c54a6e1eea68d86be2c2ff7120a51077d9816d4c67ed4071d2d9439419f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e) 2024-09-05T01:34:04.437 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:04 smithi183 podman[70238]: 2024-09-05 01:34:04.188432597 +0000 UTC m=+0.407515004 container init a63c54a6e1eea68d86be2c2ff7120a51077d9816d4c67ed4071d2d9439419f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T01:34:04.438 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:04 smithi183 podman[70238]: 2024-09-05 01:34:04.191767305 +0000 UTC m=+0.410849714 container start a63c54a6e1eea68d86be2c2ff7120a51077d9816d4c67ed4071d2d9439419f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731) 2024-09-05T01:34:04.438 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:04 smithi183 podman[70238]: 2024-09-05 01:34:04.255257676 +0000 UTC m=+0.474340082 container attach a63c54a6e1eea68d86be2c2ff7120a51077d9816d4c67ed4071d2d9439419f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T01:34:04.837 INFO:journalctl@ceph.osd.1.smithi183.stdout:Sep 05 01:34:04 smithi183 podman[70238]: 2024-09-05 01:34:04.437301914 +0000 UTC m=+0.656384322 container died a63c54a6e1eea68d86be2c2ff7120a51077d9816d4c67ed4071d2d9439419f3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T01:34:05.148 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.1.service' 2024-09-05T01:34:05.189 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:05.189 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-05T01:34:05.189 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-05T01:34:05.189 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.2 2024-09-05T01:34:05.501 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:05 smithi194 systemd[1]: Stopping Ceph osd.2 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:06.000 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:34:05.590+0000 7f0394283640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:06.001 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:34:05.590+0000 7f0394283640 -1 osd.2 97 *** Got signal Terminated *** 2024-09-05T01:34:06.001 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2[39888]: 2024-09-05T01:34:05.590+0000 7f0394283640 -1 osd.2 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:11.501 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:11 smithi194 podman[71744]: 2024-09-05 01:34:11.200644443 +0000 UTC m=+5.818310053 container died c29fa8f4e1e6590a2faf1ab4b40614d891cdfa0ee08ca222258262262fb48538 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:34:11.909 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:11 smithi194 podman[71744]: 2024-09-05 01:34:11.599369082 +0000 UTC m=+6.217034685 container cleanup c29fa8f4e1e6590a2faf1ab4b40614d891cdfa0ee08ca222258262262fb48538 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-05T01:34:11.910 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:11 smithi194 bash[71744]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2 2024-09-05T01:34:12.218 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:11 smithi194 podman[71758]: 2024-09-05 01:34:11.909316347 +0000 UTC m=+0.709034828 container remove c29fa8f4e1e6590a2faf1ab4b40614d891cdfa0ee08ca222258262262fb48538 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T01:34:12.501 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.121119022 +0000 UTC m=+0.015929535 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:12.501 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.243553939 +0000 UTC m=+0.138364441 container create ecf9e2bb3c5b6f555597a22b77472d43807b15a1a2c313c80ef3ed938d0d82cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e) 2024-09-05T01:34:12.762 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.512860651 +0000 UTC m=+0.407671156 container init ecf9e2bb3c5b6f555597a22b77472d43807b15a1a2c313c80ef3ed938d0d82cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:12.762 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.516272323 +0000 UTC m=+0.411082832 container start ecf9e2bb3c5b6f555597a22b77472d43807b15a1a2c313c80ef3ed938d0d82cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T01:34:12.762 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.612903327 +0000 UTC m=+0.507713838 container attach ecf9e2bb3c5b6f555597a22b77472d43807b15a1a2c313c80ef3ed938d0d82cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_CLEAN=True) 2024-09-05T01:34:13.118 INFO:journalctl@ceph.osd.2.smithi194.stdout:Sep 05 01:34:12 smithi194 podman[71849]: 2024-09-05 01:34:12.761777206 +0000 UTC m=+0.656587710 container died ecf9e2bb3c5b6f555597a22b77472d43807b15a1a2c313c80ef3ed938d0d82cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T01:34:13.661 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.2.service' 2024-09-05T01:34:13.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:13.703 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-05T01:34:13.703 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-05T01:34:13.703 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.3 2024-09-05T01:34:13.992 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:13 smithi129 systemd[1]: Stopping Ceph osd.3 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:14.492 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:14 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:34:14.012+0000 7f575342c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:14.493 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:14 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:34:14.012+0000 7f575342c640 -1 osd.3 97 *** Got signal Terminated *** 2024-09-05T01:34:14.493 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:14 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3[51380]: 2024-09-05T01:34:14.012+0000 7f575342c640 -1 osd.3 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:15.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:15.237+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:16.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:16 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:16.224+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:17.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:17 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:17.272+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:18.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:17.837+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:18.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:18 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:18.281+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:19.242 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:18 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:18.773+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:19.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:18.820+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:19.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:19 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:19.250+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:19.739 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:19 smithi129 podman[108679]: 2024-09-05 01:34:19.441126249 +0000 UTC m=+5.546470268 container died f488d543835db8804057b959a5e562d3ebc1bf0a976ebc4dbdb484cd0acceae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:34:19.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:19 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:19.768+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:19.993 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:19 smithi129 podman[108679]: 2024-09-05 01:34:19.937972804 +0000 UTC m=+6.043316834 container cleanup f488d543835db8804057b959a5e562d3ebc1bf0a976ebc4dbdb484cd0acceae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T01:34:19.993 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:19 smithi129 bash[108679]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3 2024-09-05T01:34:20.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:19.789+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:20.492 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108696]: 2024-09-05 01:34:20.181525166 +0000 UTC m=+0.736024159 container remove f488d543835db8804057b959a5e562d3ebc1bf0a976ebc4dbdb484cd0acceae4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T01:34:20.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:20 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:20.249+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:20.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:20.273+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:20.868 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:20 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:20.734+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:20.869 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108787]: 2024-09-05 01:34:20.508758756 +0000 UTC m=+0.015994095 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:20.869 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108787]: 2024-09-05 01:34:20.607489103 +0000 UTC m=+0.114724438 container create fbbca2cd88967f26944939ed4fc2138fa2dd8b85ab839bbb7c79e5311aba519f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-05T01:34:21.243 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108787]: 2024-09-05 01:34:20.868436287 +0000 UTC m=+0.375671625 container init fbbca2cd88967f26944939ed4fc2138fa2dd8b85ab839bbb7c79e5311aba519f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:21.243 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108787]: 2024-09-05 01:34:20.871966237 +0000 UTC m=+0.379201577 container start fbbca2cd88967f26944939ed4fc2138fa2dd8b85ab839bbb7c79e5311aba519f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e) 2024-09-05T01:34:21.243 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:20 smithi129 podman[108787]: 2024-09-05 01:34:20.976890743 +0000 UTC m=+0.484126087 container attach fbbca2cd88967f26944939ed4fc2138fa2dd8b85ab839bbb7c79e5311aba519f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-05T01:34:21.243 INFO:journalctl@ceph.osd.3.smithi129.stdout:Sep 05 01:34:21 smithi129 podman[108787]: 2024-09-05 01:34:21.067324504 +0000 UTC m=+0.574559847 container died fbbca2cd88967f26944939ed4fc2138fa2dd8b85ab839bbb7c79e5311aba519f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-3-deactivate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:21.248 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:20.813+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:21.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:21.247+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:21.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:21 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:21.252+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:21.679 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.3.service' 2024-09-05T01:34:21.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:21.718 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-05T01:34:21.719 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-05T01:34:21.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.4 2024-09-05T01:34:21.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:21 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:21.719+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:22.045 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:21 smithi183 systemd[1]: Stopping Ceph osd.4 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:22.231 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:21.815+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:22.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:22 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:22.243+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:22.411 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:22 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:34:22.043+0000 7f7f86a3c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:22.411 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:22 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:34:22.043+0000 7f7f86a3c640 -1 osd.4 97 *** Got signal Terminated *** 2024-09-05T01:34:22.411 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:22 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4[45652]: 2024-09-05T01:34:22.043+0000 7f7f86a3c640 -1 osd.4 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:22.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:22.230+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:22.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:22 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:22.726+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:23.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:22.776+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:23.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:23 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:23.287+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:23.750 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:23.272+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:23.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:23.511+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:23.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:23 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:23.740+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:23.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:23 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:23.740+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:24.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:23.804+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:24.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:24 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:24.243+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:24.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:24 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:24.185+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:24.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:24.486+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:24.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:24.313+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:25.161 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:24 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:24.755+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:25.161 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:24 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:24.755+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:25.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:24.822+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:25.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:24 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:24.822+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:25.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:25 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:25.205+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:25.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:25 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:25.267+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:25.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:25 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:25.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:25.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:25 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:25.275+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:26.121 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:25.741+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:26.121 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:25 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:25.741+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:26.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:25 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:25.847+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:26.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:25 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:25.847+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:26.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:26 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:26.286+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:26.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:26 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:26.251+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:26.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:26.483+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:26.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:26.286+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:26.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:26.286+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:26.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:26 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:26.702+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:26.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:26 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:26.702+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:27.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:26.871+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:27.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:26 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:26.871+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:27.575 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:27 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:27.226+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:27.576 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:27 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:27.236+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:27.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:27 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:27.488+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:27.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:27 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:27.272+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:27.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:27 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:27.272+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:27.878 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:27 smithi183 podman[70353]: 2024-09-05 01:34:27.575367183 +0000 UTC m=+5.647858299 container died 480cfa2589ccc0f3e08221eb002e15260f53f75ceb0c3b9e0f873f827cd185c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T01:34:27.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:27 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:27.662+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:27.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:27 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:27.662+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:28.161 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:27 smithi183 podman[70353]: 2024-09-05 01:34:27.977235311 +0000 UTC m=+6.049726416 container cleanup 480cfa2589ccc0f3e08221eb002e15260f53f75ceb0c3b9e0f873f827cd185c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-05T01:34:28.161 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:27 smithi183 bash[70353]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4 2024-09-05T01:34:28.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:27 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:27.890+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:28.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:27 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:27.890+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:28.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:28 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:28.038+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:28.531 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:28 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:28.277+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:28.531 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:28 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:28.277+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:28.593 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:28 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:28.185+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:28.593 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70367]: 2024-09-05 01:34:28.245388699 +0000 UTC m=+0.664976579 container remove 480cfa2589ccc0f3e08221eb002e15260f53f75ceb0c3b9e0f873f827cd185c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T01:34:28.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:28 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:28.279+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:28.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:28 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:28.279+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:28.875 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70457]: 2024-09-05 01:34:28.495210935 +0000 UTC m=+0.016118172 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:28.875 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70457]: 2024-09-05 01:34:28.625208864 +0000 UTC m=+0.146116089 container create 7a6172f12839953cedf8b623570cc5f2b65eefef4f388d492ebfb107f113628c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-05T01:34:28.922 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:28 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:28.530+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:28.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:28 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:28.659+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:28.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:28 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:28.659+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:29.161 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70457]: 2024-09-05 01:34:28.875073518 +0000 UTC m=+0.395980745 container init 7a6172f12839953cedf8b623570cc5f2b65eefef4f388d492ebfb107f113628c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-05T01:34:29.161 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70457]: 2024-09-05 01:34:28.878650481 +0000 UTC m=+0.399557710 container start 7a6172f12839953cedf8b623570cc5f2b65eefef4f388d492ebfb107f113628c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-05T01:34:29.161 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:28 smithi183 podman[70457]: 2024-09-05 01:34:28.982551259 +0000 UTC m=+0.503458492 container attach 7a6172f12839953cedf8b623570cc5f2b65eefef4f388d492ebfb107f113628c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T01:34:29.162 INFO:journalctl@ceph.osd.4.smithi183.stdout:Sep 05 01:34:29 smithi183 podman[70457]: 2024-09-05 01:34:29.064536358 +0000 UTC m=+0.585443580 container died 7a6172f12839953cedf8b623570cc5f2b65eefef4f388d492ebfb107f113628c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T01:34:29.252 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:28 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:28.920+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:29.252 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:28 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:28.920+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:29.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:29 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:29.062+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:29.550 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:29.285+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:29.551 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:29.285+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:29.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:29.224+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:29.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:29.322+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:29.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:29 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:29.322+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:29.738 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.4.service' 2024-09-05T01:34:29.776 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:29.776 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-05T01:34:29.777 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-05T01:34:29.777 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.5 2024-09-05T01:34:29.804 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:29.548+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:29.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:29 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:29.691+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:29.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:29 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:29.691+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:30.120 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:29 smithi194 systemd[1]: Stopping Ceph osd.5 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:30.120 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:29.965+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:30.121 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:29 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:29.965+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:30.121 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:30.118+0000 7fb3a798f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:30.121 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:30.118+0000 7fb3a798f640 -1 osd.5 97 *** Got signal Terminated *** 2024-09-05T01:34:30.121 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:30.118+0000 7fb3a798f640 -1 osd.5 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:30.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:30.057+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:30.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:30.266+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:30.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:30.266+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:30.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:30 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:30.184+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:30.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:30 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:30.277+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:30.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:30 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:30.277+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:30.954 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:30.510+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:30.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:30.718+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:30.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:30.718+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:31.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:30.953+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:31.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:30 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:30.953+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:31.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:31 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:31.151+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:31.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:31 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:31.271+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:31.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:31 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:31.271+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:31.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:31 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:31.082+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:31.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:31 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:31.498+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:31.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:31 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:31.286+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:31.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:31 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:31.286+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:31.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:31 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:31.669+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:31.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:31 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:31.669+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:32.250 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:31 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:31.919+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:32.251 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:31 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:31.919+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:32.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:32 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:32.106+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:32.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:32 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:32.282+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:32.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:32 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:32.282+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:32.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:32 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:32.123+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:32.520 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:32.251+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:32.520 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:32.251+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:32.520 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:32.251+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:32.939 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:32.519+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:32.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:32 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:32.641+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:32.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:32 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:32.641+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:33.215 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:32.937+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:33.215 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:32 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:32.937+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:33.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:33 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:33.155+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:33.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:33 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:33.288+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:33.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:33 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:33.288+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:33.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:33 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:33.128+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:33.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:33 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:33.128+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:33.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:33.214+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:33.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:33.214+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:33.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:33.214+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:33.901 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:33.558+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:33.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:33 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:33.690+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:33.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:33 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:33.690+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:33.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:33 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:33.690+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:34.180 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:33.899+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:34.180 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:33 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:33.899+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:34.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:34 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:34.110+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:34.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:34 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:34.335+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:34.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:34 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:34.335+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:34.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:34 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:34.335+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:34.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:34.117+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:34.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:34.117+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:34.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:34.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:34.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:34.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:34.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:34.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:34.855 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:34.569+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:34.855 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:34.569+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:34.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:34.720+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:34.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:34.720+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:34.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:34 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:34.720+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:35.148 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:34.854+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.322150+0000 front 2024-09-05T01:33:53.322088+0000 (oldest deadline 2024-09-05T01:34:17.422128+0000) 2024-09-05T01:34:35.148 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:34.854+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.923349+0000 front 2024-09-05T01:34:00.923363+0000 (oldest deadline 2024-09-05T01:34:23.823445+0000) 2024-09-05T01:34:35.148 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:34 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5[46790]: 2024-09-05T01:34:34.854+0000 7fb39ff9d640 -1 osd.5 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.225194+0000 front 2024-09-05T01:34:10.225220+0000 (oldest deadline 2024-09-05T01:34:34.325355+0000) 2024-09-05T01:34:35.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:35 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:35.127+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:35.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:35 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:35.127+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:35.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:35 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:35.339+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:35.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:35 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:35.339+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:35.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:35 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:35.339+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:35.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:35.154+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:35.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:35.154+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:35.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:35 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:35.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:35.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:35 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:35.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:35.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:35 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:35.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:35.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:35.691+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:35.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:35.691+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:35.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:35.691+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:36.000 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:35 smithi194 podman[71961]: 2024-09-05 01:34:35.742543649 +0000 UTC m=+5.764615185 container died 41b5e1e3f123f8fa636949fe20b48f208cbf2aa2ae95b199f0172fc49ebb4298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:34:36.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:35 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:35.552+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:36.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:35 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:35.552+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:36.369 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:36 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:36.116+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:36.369 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:36 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:36.116+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:36.405 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:36 smithi194 podman[71961]: 2024-09-05 01:34:36.136876849 +0000 UTC m=+6.158948430 container cleanup 41b5e1e3f123f8fa636949fe20b48f208cbf2aa2ae95b199f0172fc49ebb4298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:34:36.405 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:36 smithi194 bash[71961]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5 2024-09-05T01:34:36.405 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:36 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:36.109+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:36.405 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:36 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:36.109+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:36.405 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:36 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:36.109+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:36.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:36.120+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:36.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:36.120+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:36.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:36 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:36.367+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:36.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:36 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:36.367+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:36.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:36 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:36.367+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:36.750 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:36 smithi194 podman[71976]: 2024-09-05 01:34:36.404838997 +0000 UTC m=+0.666215359 container remove 41b5e1e3f123f8fa636949fe20b48f208cbf2aa2ae95b199f0172fc49ebb4298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T01:34:36.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:36 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:36.505+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:36.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:36 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:36.505+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:36.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:36.644+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:36.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:36.644+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:36.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:36 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:36.644+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:37.112 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:36 smithi194 podman[72062]: 2024-09-05 01:34:36.718299245 +0000 UTC m=+0.016138577 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:37.112 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:36 smithi194 podman[72062]: 2024-09-05 01:34:36.855581497 +0000 UTC m=+0.153420808 container create c86e677912f201117d0d93b12a34a8718486bbcdf3adbc72038438b2a3819a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:37 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:37.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:37 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:37.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:37 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:37.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:37 smithi194 podman[72062]: 2024-09-05 01:34:37.150710282 +0000 UTC m=+0.448549594 container init c86e677912f201117d0d93b12a34a8718486bbcdf3adbc72038438b2a3819a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:37 smithi194 podman[72062]: 2024-09-05 01:34:37.154257298 +0000 UTC m=+0.452096616 container start c86e677912f201117d0d93b12a34a8718486bbcdf3adbc72038438b2a3819a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:37 smithi194 podman[72062]: 2024-09-05 01:34:37.249955139 +0000 UTC m=+0.547794452 container attach c86e677912f201117d0d93b12a34a8718486bbcdf3adbc72038438b2a3819a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-05T01:34:37.477 INFO:journalctl@ceph.osd.5.smithi194.stdout:Sep 05 01:34:37 smithi194 podman[72062]: 2024-09-05 01:34:37.338156802 +0000 UTC m=+0.635996127 container died c86e677912f201117d0d93b12a34a8718486bbcdf3adbc72038438b2a3819a0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-05T01:34:37.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:37.087+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:37.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:37.087+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:37.532 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:37.165+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:37.533 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:37.165+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:37.533 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:37.357+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:37.533 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:37.357+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:37.533 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:37 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:37.358+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:37.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:37 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:37.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:37.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:37 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:37.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:37.962 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.5.service' 2024-09-05T01:34:37.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:37.675+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:37.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:37.675+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:37.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:37 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:37.675+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:38.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:38.040 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-05T01:34:38.040 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-05T01:34:38.041 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.6 2024-09-05T01:34:38.250 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:38.108+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:38.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:38.108+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:38.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:38.108+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 systemd[1]: Stopping Ceph osd.6 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:38.114+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:38.114+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:38.366+0000 7f23f6cab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:38.366+0000 7f23f6cab640 -1 osd.6 97 *** Got signal Terminated *** 2024-09-05T01:34:38.368 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:38.366+0000 7f23f6cab640 -1 osd.6 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:38.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:38 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:38.146+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:38.502 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:38 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:38.146+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:38.502 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:38 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:38.361+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:38.502 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:38 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:38.361+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:38.502 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:38 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:38.361+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:38.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:38.484+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:38.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:38 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:38.484+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:38.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:38.627+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:38.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:38.627+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:38.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:38 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:38.627+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:39.401 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:39 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:39.107+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:39.401 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:39 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:39.107+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:39.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:39.099+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:39.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:39.099+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:39.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:39.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:39.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:39.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:39.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:39.146+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:39.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:39 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:39.400+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:39.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:39 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:39.400+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:39.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:39 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:39.400+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:39.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:39.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:39.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:39.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:39.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:39 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:39.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:40.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:39.500+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:40.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:39 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:39.500+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:40.404 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:40 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:40.083+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:40.404 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:40 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:40.083+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:40.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:40.094+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:40.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:40.094+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:40.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:40 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:40.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:40.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:40 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:40.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:40.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:40 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:40.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:40.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:40 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:40.403+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:40.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:40 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:40.403+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:40.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:40 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:40.403+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:40.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:40.655+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:40.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:40.655+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:40.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:40.655+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:41.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:40 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:40.522+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:41.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:40 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:40.522+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:41.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:41.106+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:41.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:41.106+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:41.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:41.106+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:41.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:41.045+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:41.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:41.045+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:41.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:41.197+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:41.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:41.197+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:41.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:41.197+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:41.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:41.430+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:41.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:41.430+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:41.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:41 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:41.430+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:41.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:41.625+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:41.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:41.625+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:41.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:41 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:41.625+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:42.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:41.526+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:42.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:41 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:41.526+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:42.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:42.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:42.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:42.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:42.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:42.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:42.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:42.048+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:42.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:42.048+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:42.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:42.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:42.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:42.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:42.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:42.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:42.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:42.396+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:42.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:42.396+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:42.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:42.396+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:42.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:42 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:42.396+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:42.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:42.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:42.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:42.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:42.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:42.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:42.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:42 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:42.612+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:43.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:42.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:43.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:42 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:42.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:43.396 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:43.079+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:43.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:43.079+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:43.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:43.079+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:43.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:43.028+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.694021+0000 front 2024-09-05T01:34:01.694093+0000 (oldest deadline 2024-09-05T01:34:27.593964+0000) 2024-09-05T01:34:43.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:43.028+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.895222+0000 front 2024-09-05T01:34:09.895252+0000 (oldest deadline 2024-09-05T01:34:32.795349+0000) 2024-09-05T01:34:43.492 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6[58052]: 2024-09-05T01:34:43.028+0000 7f23ef2b9640 -1 osd.6 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.796737+0000 front 2024-09-05T01:34:16.796759+0000 (oldest deadline 2024-09-05T01:34:42.096887+0000) 2024-09-05T01:34:43.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:43.148+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:43.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:43.148+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:43.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:43.148+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:43.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:43.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:43.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:43.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:43.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:43.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:43.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:43.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:43.959 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:43.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:43.959 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:43.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:43.959 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:43.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:43.959 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:43 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:43.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:44.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:43.517+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:44.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:43.517+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:44.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:43 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:43.517+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:44.242 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:43 smithi129 podman[108901]: 2024-09-05 01:34:43.958713029 +0000 UTC m=+5.720675932 container died 30d225fc91e49eb7d82d5d3a36802c3120f099a72396cf26b9b39f7e82ec2c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2) 2024-09-05T01:34:44.276 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:44.179+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:44.276 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:44.179+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:44.276 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:44.179+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:44.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:44.050+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:44.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:44.050+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:44.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:44.050+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:44.563 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:44 smithi129 podman[108901]: 2024-09-05 01:34:44.450440901 +0000 UTC m=+6.212403845 container cleanup 30d225fc91e49eb7d82d5d3a36802c3120f099a72396cf26b9b39f7e82ec2c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux ) 2024-09-05T01:34:44.563 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:44 smithi129 bash[108901]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6 2024-09-05T01:34:44.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:44.429+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:44.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:44.429+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:44.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:44.429+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:44.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:44 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:44.429+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:44.992 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:44 smithi129 podman[108921]: 2024-09-05 01:34:44.760349634 +0000 UTC m=+0.800348981 container remove 30d225fc91e49eb7d82d5d3a36802c3120f099a72396cf26b9b39f7e82ec2c9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T01:34:44.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:44.561+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:44.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:44.561+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:44.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:44.561+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:44.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:44 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:44.561+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:45.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:44.561+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:45.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:44.561+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:45.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:44 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:44.561+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:45.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:45.059+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:45.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:45.059+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:45.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:45.059+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:45.465 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.02095256 +0000 UTC m=+0.015821662 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:45.465 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.127939036 +0000 UTC m=+0.122808128 container create a957749a3d9d2facd2e72ee66a710e011d69080bc55ddd0c53ebc127dc37d034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:45.465 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.363963691 +0000 UTC m=+0.358832784 container init a957749a3d9d2facd2e72ee66a710e011d69080bc55ddd0c53ebc127dc37d034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-05T01:34:45.465 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.36722725 +0000 UTC m=+0.362096343 container start a957749a3d9d2facd2e72ee66a710e011d69080bc55ddd0c53ebc127dc37d034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T01:34:45.465 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.463990216 +0000 UTC m=+0.458859316 container attach a957749a3d9d2facd2e72ee66a710e011d69080bc55ddd0c53ebc127dc37d034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:34:45.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:45.220+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:45.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:45.220+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:45.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:45.220+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:45.742 INFO:journalctl@ceph.osd.6.smithi129.stdout:Sep 05 01:34:45 smithi129 podman[109006]: 2024-09-05 01:34:45.555305996 +0000 UTC m=+0.550175092 container died a957749a3d9d2facd2e72ee66a710e011d69080bc55ddd0c53ebc127dc37d034 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-05T01:34:45.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:45 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:45.551+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:45.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:45 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:45.551+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:45.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:45 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:45.551+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:45.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:45 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:45.551+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:45.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:45.415+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:45.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:45.415+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:45.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:45.415+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:45.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:45 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:45.415+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:46.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:45.551+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:46.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:45.551+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:46.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:45 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:45.551+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:46.199 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.6.service' 2024-09-05T01:34:46.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:46.240 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-05T01:34:46.241 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-05T01:34:46.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.7 2024-09-05T01:34:46.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:46.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:46.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:46.397 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 systemd[1]: Stopping Ceph osd.7 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:46.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:46.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:46.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:46.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:46.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:46.178+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:46.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.554+0000 7fc626172640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:46.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.554+0000 7fc626172640 -1 osd.7 97 *** Got signal Terminated *** 2024-09-05T01:34:46.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:46.554+0000 7fc626172640 -1 osd.7 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:46.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:46.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:46.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:46.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:46.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:46.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:46.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:46.395+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:46.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:46 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:46.523+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:46.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:46 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:46.523+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:46.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:46 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:46.523+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:46.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:46 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:46.523+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:47.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:46.504+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:47.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:46.504+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:47.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:46 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:46.504+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:47.353 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:47.109+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:47.353 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:47.109+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:47.353 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:47.109+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:47.473 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:47.222+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:47.473 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:47.222+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:47.473 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:47.222+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:47.609 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:47.351+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:47.609 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:47.351+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:47.609 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:47.351+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:47.609 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:47 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:47.351+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:47.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:47 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:47.513+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:47.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:47 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:47.513+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:47.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:47 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:47.513+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:47.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:47 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:47.513+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:47.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:47.472+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:47.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:47.472+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:47.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:47 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:47.472+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:48.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:48.141+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:48.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:48.141+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:48.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:48.141+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:48.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:48.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:48.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:48.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:48.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:48.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:48.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:48 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:48.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:48.750 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:48.259+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:48.259+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:48.259+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:48.467+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:48.467+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:48.467+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:48.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:48 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:48.467+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:48.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:48 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:48.563+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:48.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:48 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:48.563+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:48.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:48 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:48.563+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:48.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:48 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:48.563+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:49.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:49.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:49.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:49.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:49.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:49.095+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:49.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:49.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:49.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:49.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:49.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:49.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:49.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:49 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:49.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:49.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:49.482+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:49.482+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:49.482+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:49.482+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:49.255+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:49.255+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:49.255+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:49.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:49 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:49.255+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:49.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:49.517+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:49.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:49.517+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:49.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:49.517+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:49.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:49 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:49.517+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:50.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:50.078+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:50.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:50.078+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:50.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:50.078+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:50.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:50.326+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:50.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:50.326+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:50.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:50.326+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:50.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:50.326+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:50.435+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:50.435+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:50.435+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:50.435+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:50.252+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:50.252+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:50.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:50.252+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:50.752 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:50 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:50.252+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:50.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:50.503+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:50.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:50.503+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:50.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:50.503+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:50.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:50 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:50.503+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:51.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:51.105+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.971928+0000 front 2024-09-05T01:33:58.971956+0000 (oldest deadline 2024-09-05T01:34:23.672076+0000) 2024-09-05T01:34:51.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:51.105+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.874493+0000 front 2024-09-05T01:34:09.874378+0000 (oldest deadline 2024-09-05T01:34:34.574403+0000) 2024-09-05T01:34:51.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:51.105+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:18.675740+0000 front 2024-09-05T01:34:18.675800+0000 (oldest deadline 2024-09-05T01:34:40.975538+0000) 2024-09-05T01:34:51.411 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7[52269]: 2024-09-05T01:34:51.105+0000 7fc61e780640 -1 osd.7 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.477362+0000 front 2024-09-05T01:34:25.477386+0000 (oldest deadline 2024-09-05T01:34:50.177590+0000) 2024-09-05T01:34:51.411 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:51.320+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:51.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:51.320+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:51.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:51.320+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:51.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:51.320+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:51.412 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:51 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:51.320+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:51.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:51.216+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:51.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:51.216+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:51.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:51.216+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:51.476 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:51.216+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:51.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:51.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:51.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:51.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:51.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:51.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:51.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:51 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:51.475+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:51.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:51.542+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:51.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:51.542+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:51.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:51.542+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:51.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:51.542+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:51.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:51 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:51.542+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:52.495 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:52.205+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:52.496 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:52.205+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:52.496 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:52.205+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:52.496 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:52.205+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:52.505 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:52 smithi183 podman[70574]: 2024-09-05 01:34:52.205077573 +0000 UTC m=+5.762850133 container died 06485557bfeaa011f8e2a546443ccf4ae6cf071be69c6b3344608d5d4e24b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-05T01:34:52.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:52 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:52.310+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:52.505 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:52 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:52.310+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:52.506 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:52 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:52.310+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:52.506 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:52 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:52.310+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:52.506 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:52 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:52.310+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:52.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:52.495+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:52.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:52.495+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:52.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:52.495+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:52.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:52 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:52.495+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:52.839 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:52 smithi183 podman[70574]: 2024-09-05 01:34:52.595962183 +0000 UTC m=+6.153734744 container cleanup 06485557bfeaa011f8e2a546443ccf4ae6cf071be69c6b3344608d5d4e24b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:52.839 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:52 smithi183 bash[70574]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7 2024-09-05T01:34:52.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:52.584+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:52.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:52.584+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:52.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:52.584+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:52.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:52.584+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:52.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:52 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:52.584+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:53.147 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:52 smithi183 podman[70589]: 2024-09-05 01:34:52.838718593 +0000 UTC m=+0.632996264 container remove 06485557bfeaa011f8e2a546443ccf4ae6cf071be69c6b3344608d5d4e24b905 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-05T01:34:53.408 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.049985957 +0000 UTC m=+0.016196528 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:34:53.408 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.147748929 +0000 UTC m=+0.113959488 container create 0f0e8e0298d9ced8f9103f7bade6ca8699be9dca8e05ed16a073f99f9868eb61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:34:53.409 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:53 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:53.319+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:53.409 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:53 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:53.319+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:53.409 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:53 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:53.319+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:53.409 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:53 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:53.319+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:53.409 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:53 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:53.319+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:53.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:53.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:53.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:53.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:53.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:53.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:53.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:53.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:53.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.408421641 +0000 UTC m=+0.374632192 container init 0f0e8e0298d9ced8f9103f7bade6ca8699be9dca8e05ed16a073f99f9868eb61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2) 2024-09-05T01:34:53.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.411806077 +0000 UTC m=+0.378016632 container start 0f0e8e0298d9ced8f9103f7bade6ca8699be9dca8e05ed16a073f99f9868eb61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-05T01:34:53.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.516817425 +0000 UTC m=+0.483027994 container attach 0f0e8e0298d9ced8f9103f7bade6ca8699be9dca8e05ed16a073f99f9868eb61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-05T01:34:53.661 INFO:journalctl@ceph.osd.7.smithi183.stdout:Sep 05 01:34:53 smithi183 podman[70678]: 2024-09-05 01:34:53.600306903 +0000 UTC m=+0.566517465 container died 0f0e8e0298d9ced8f9103f7bade6ca8699be9dca8e05ed16a073f99f9868eb61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-05T01:34:53.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:53.554+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:53.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:53.554+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:53.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:53.554+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:53.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:53.554+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:53.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:53 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:53.554+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:54.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:53.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:54.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:53.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:54.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:53.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:54.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:53 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:53.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:54.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.7.service' 2024-09-05T01:34:54.252 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:34:54.252 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-05T01:34:54.252 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-05T01:34:54.252 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.8 2024-09-05T01:34:54.461 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:54 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:54.352+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:54.461 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:54 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:54.352+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:54.461 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:54 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:54.352+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:54.461 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:54 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:54.352+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:54.461 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:54 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:54.352+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:54.501 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 systemd[1]: Stopping Ceph osd.8 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:34:54.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:54.198+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:54.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:54.198+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:54.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:54.198+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:54.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:54.198+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:54.894 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:54.514+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:54.894 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:54.514+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:54.894 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:54.514+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:54.894 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:54.514+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:54.894 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:54 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:54.514+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:55.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.545+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.545+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.545+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.545+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.598+0000 7fe640ada640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.598+0000 7fe640ada640 -1 osd.8 97 *** Got signal Terminated *** 2024-09-05T01:34:55.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:54 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:54.598+0000 7fe640ada640 -1 osd.8 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:34:55.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:55.181+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:55.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:55.181+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:55.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:55.181+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:55.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:55.181+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:55.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:55.328+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:55.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:55.328+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:55.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:55.328+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:55.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:55.328+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:55.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:55.328+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:55.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:55 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:55.534+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:55.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:55 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:55.534+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:55.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:55 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:55.534+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:55.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:55 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:55.534+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:55.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:55 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:55.534+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:56.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:55.524+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:56.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:55.524+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:56.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:55.524+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:56.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:55 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:55.524+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:56.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:56.168+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:56.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:56.168+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:56.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:56.168+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:56.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:56.168+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:56.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:56.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:56.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:56.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:56.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:56.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:56.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:56.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:56.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:56 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:56.345+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:56.916 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:56.496+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:56.916 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:56.496+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:56.917 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:56.496+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:56.917 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:56.496+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:56.917 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:56 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:56.496+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:57.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:56.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:57.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:56.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:57.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:56.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:57.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:56.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:57.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:56 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:56.527+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.238019+0000 front 2024-09-05T01:34:34.238055+0000 (oldest deadline 2024-09-05T01:34:55.938096+0000) 2024-09-05T01:34:57.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:57.147+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:57.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:57.147+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:57.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:57.147+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:57.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:57.147+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:57.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:57.147+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:34:57.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:57.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:57.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:57.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:57.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:57.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:57 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:57.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:34:57.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:57 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:57.530+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:57.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:57 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:57.530+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:57.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:57 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:57.530+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:57.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:57 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:57.530+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:57.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:57 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:57.530+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:58.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:57.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:58.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:57.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:58.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:57.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:58.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:57.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:58.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:57 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:57.518+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.238019+0000 front 2024-09-05T01:34:34.238055+0000 (oldest deadline 2024-09-05T01:34:55.938096+0000) 2024-09-05T01:34:58.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:58.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:58.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:58.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:58.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:58.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:58.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:58.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:58.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:58.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:34:58.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:58.923 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:58.923 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:58.923 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:58.923 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:58.923 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:58 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:58.412+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:34:58.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:58 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:58.557+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:58.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:58 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:58.557+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:58.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:58 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:58.557+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:58.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:58 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:58.557+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:58.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:58 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:58.557+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:34:59.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:58.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:59.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:58.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:59.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:58.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:59.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:58.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:59.001 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:58 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:58.502+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.238019+0000 front 2024-09-05T01:34:34.238055+0000 (oldest deadline 2024-09-05T01:34:55.938096+0000) 2024-09-05T01:34:59.497 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:59.127+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:34:59.497 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:59.127+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:34:59.498 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:59.127+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:34:59.498 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:59.127+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:34:59.498 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:34:59.127+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:34:59.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:59.496+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:01.431325+0000 front 2024-09-05T01:34:01.431498+0000 (oldest deadline 2024-09-05T01:34:23.131286+0000) 2024-09-05T01:34:59.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:59.496+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.334192+0000 front 2024-09-05T01:34:09.334062+0000 (oldest deadline 2024-09-05T01:34:34.034187+0000) 2024-09-05T01:34:59.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:59.496+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.335954+0000 front 2024-09-05T01:34:17.336063+0000 (oldest deadline 2024-09-05T01:34:42.635978+0000) 2024-09-05T01:34:59.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:59.496+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.737339+0000 front 2024-09-05T01:34:26.737153+0000 (oldest deadline 2024-09-05T01:34:48.436977+0000) 2024-09-05T01:34:59.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:34:59 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8[53418]: 2024-09-05T01:34:59.496+0000 7fe6390e8640 -1 osd.8 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.238019+0000 front 2024-09-05T01:34:34.238055+0000 (oldest deadline 2024-09-05T01:34:55.938096+0000) 2024-09-05T01:34:59.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:34:59.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:34:59.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:34:59.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:34:59.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:34:59.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:34:59 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:34:59.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:34:59.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:59.591+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:34:59.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:59.591+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:34:59.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:59.591+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:34:59.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:59.591+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:34:59.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:34:59 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:34:59.591+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:00.374 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:00 smithi194 podman[72177]: 2024-09-05 01:35:00.091518488 +0000 UTC m=+5.645746480 container died ae8448de07fe0d53d232c39e504ac43edcda14dbf28bb38ce4e9db7b56656e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-05T01:35:00.374 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:00.149+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:00.374 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:00.149+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:00.374 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:00.149+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:00.374 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:00.149+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:00.375 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:00.149+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:00.740 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:00 smithi194 podman[72177]: 2024-09-05 01:35:00.47230379 +0000 UTC m=+6.026531782 container cleanup ae8448de07fe0d53d232c39e504ac43edcda14dbf28bb38ce4e9db7b56656e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T01:35:00.740 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:00 smithi194 bash[72177]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8 2024-09-05T01:35:00.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:00.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:00.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:00.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:00.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:00.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:00 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:00.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:00.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:00.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:00.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:00.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:00.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:00.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:00 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:00.624+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:01.000 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:00 smithi194 podman[72193]: 2024-09-05 01:35:00.740232179 +0000 UTC m=+0.644573181 container remove ae8448de07fe0d53d232c39e504ac43edcda14dbf28bb38ce4e9db7b56656e11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-05T01:35:01.312 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:00.951887221 +0000 UTC m=+0.015891333 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:35:01.312 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:01.068128738 +0000 UTC m=+0.132132841 container create 55e6010b470f492f0eca322af4aabe27ba32408ca4000278c3c58b95b3f0d8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-91c8a7e) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:01.312053333 +0000 UTC m=+0.376057437 container init 55e6010b470f492f0eca322af4aabe27ba32408ca4000278c3c58b95b3f0d8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:01 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:01.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:01 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:01.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:01 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:01.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:01 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:01.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:01.313 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:01 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:01.194+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:01.750 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:01.316672091 +0000 UTC m=+0.380676197 container start 55e6010b470f492f0eca322af4aabe27ba32408ca4000278c3c58b95b3f0d8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-05T01:35:01.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:01.395363306 +0000 UTC m=+0.459367410 container attach 55e6010b470f492f0eca322af4aabe27ba32408ca4000278c3c58b95b3f0d8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8-deactivate, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T01:35:01.751 INFO:journalctl@ceph.osd.8.smithi194.stdout:Sep 05 01:35:01 smithi194 podman[72286]: 2024-09-05 01:35:01.503797352 +0000 UTC m=+0.567801463 container died 55e6010b470f492f0eca322af4aabe27ba32408ca4000278c3c58b95b3f0d8f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-05T01:35:01.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:01.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:01.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:01.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:01.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:01.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:01 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:01.444+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:01.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:01.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:01.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:01.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:01.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:01.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:01 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:01.600+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:02.168 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.8.service' 2024-09-05T01:35:02.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:02.208 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-05T01:35:02.208 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-05T01:35:02.208 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.9 2024-09-05T01:35:02.418 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:02 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:02.154+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:02.418 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:02 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:02.154+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:02.418 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:02 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:02.154+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:02.418 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:02 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:02.154+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:02.418 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:02 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:02.154+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:02.492 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 systemd[1]: Stopping Ceph osd.9 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:02.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:02 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:02.437+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:02.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.537+0000 7f2810d3f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:02.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.537+0000 7f2810d3f640 -1 osd.9 97 *** Got signal Terminated *** 2024-09-05T01:35:02.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.537+0000 7f2810d3f640 -1 osd.9 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:02.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:02 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:02.587+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:03.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:03.184+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:03.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:03.184+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:03.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:03.184+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:03.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:03.184+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:03.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:03.184+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:03.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:03 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:03.438+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:03.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:03.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:03.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:03.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:03.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:03.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:03 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:03.590+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:04.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:04 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:04.166+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:04.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:04 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:04.166+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:04.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:04 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:04.166+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:04.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:04 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:04.166+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:04.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:04 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:04.166+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:04.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:04 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:04.484+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:04.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:04.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:04.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:04.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:04.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:04.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:04 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:04.629+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:05.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:05.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:05.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:05.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:05.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:05.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:05.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:05.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:05.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:05 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:05.165+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:05.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:05 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:05.458+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:05.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:05.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:05.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:05.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:05.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:05.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:05 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:05.609+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:06.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:06.150+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:06.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:06.150+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:06.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:06.150+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:06.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:06.150+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:06.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:06.150+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:06.813 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:06.813 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:06.813 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:06.814 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:06.814 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:06.814 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:06.814 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:06 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:06.463+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:06.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:53.811808+0000 front 2024-09-05T01:33:53.811776+0000 (oldest deadline 2024-09-05T01:34:18.511981+0000) 2024-09-05T01:35:06.992 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:34:00.813015+0000 front 2024-09-05T01:34:00.812973+0000 (oldest deadline 2024-09-05T01:34:23.113002+0000) 2024-09-05T01:35:06.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:07.714796+0000 front 2024-09-05T01:34:07.714770+0000 (oldest deadline 2024-09-05T01:34:33.515437+0000) 2024-09-05T01:35:06.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:17.016495+0000 front 2024-09-05T01:34:17.016533+0000 (oldest deadline 2024-09-05T01:34:41.716675+0000) 2024-09-05T01:35:06.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:27.017841+0000 front 2024-09-05T01:34:27.017957+0000 (oldest deadline 2024-09-05T01:34:50.517720+0000) 2024-09-05T01:35:06.993 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:06 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9[64731]: 2024-09-05T01:35:06.581+0000 7f280d355640 -1 osd.9 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:34.618945+0000 front 2024-09-05T01:34:34.618925+0000 (oldest deadline 2024-09-05T01:34:59.918670+0000) 2024-09-05T01:35:07.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:07 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:07.172+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:07.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:07 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:07.172+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:07.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:07 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:07.172+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:07.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:07 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:07.172+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:07.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:07 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:07.172+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:07.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:07 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:07.464+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:08.306 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:08 smithi129 podman[109122]: 2024-09-05 01:35:08.032804817 +0000 UTC m=+5.629771867 container died ec662db70147de00e1fa72c60400815c913efe3953f1c2fc604378b136fb2e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:35:08.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:08 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:08.187+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:08.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:08 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:08.187+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:08.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:08 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:08.187+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:08.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:08 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:08.187+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:08.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:08 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:08.187+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:08.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:08 smithi129 podman[109122]: 2024-09-05 01:35:08.512005681 +0000 UTC m=+6.108972719 container cleanup ec662db70147de00e1fa72c60400815c913efe3953f1c2fc604378b136fb2e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:35:08.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:08 smithi129 bash[109122]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:08.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:08 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:08.435+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:09.101 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:08 smithi129 podman[109138]: 2024-09-05 01:35:08.790831277 +0000 UTC m=+0.761957009 container remove ec662db70147de00e1fa72c60400815c913efe3953f1c2fc604378b136fb2e89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T01:35:09.392 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.0046785 +0000 UTC m=+0.016103768 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:35:09.392 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.147657344 +0000 UTC m=+0.159082599 container create d1dee667e65e95df9f7478ef9563f923fb4115e9d048903d17ef1e6e38a3fcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS) 2024-09-05T01:35:09.392 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.391972013 +0000 UTC m=+0.403397270 container init d1dee667e65e95df9f7478ef9563f923fb4115e9d048903d17ef1e6e38a3fcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T01:35:09.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:09.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:09.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:09.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:09.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:09.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:09 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:09.227+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:09.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:09.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:09.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:09.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:09.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:09.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:09.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:09 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:09.401+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:09.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.395184405 +0000 UTC m=+0.406609670 container start d1dee667e65e95df9f7478ef9563f923fb4115e9d048903d17ef1e6e38a3fcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-05T01:35:09.742 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.491913695 +0000 UTC m=+0.503338966 container attach d1dee667e65e95df9f7478ef9563f923fb4115e9d048903d17ef1e6e38a3fcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-05T01:35:09.743 INFO:journalctl@ceph.osd.9.smithi129.stdout:Sep 05 01:35:09 smithi129 podman[109226]: 2024-09-05 01:35:09.583906736 +0000 UTC m=+0.595332001 container died d1dee667e65e95df9f7478ef9563f923fb4115e9d048903d17ef1e6e38a3fcc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T01:35:10.240 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.9.service' 2024-09-05T01:35:10.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:10.281 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-05T01:35:10.281 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-05T01:35:10.281 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.10 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:10.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:10 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:10.208+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 systemd[1]: Stopping Ceph osd.10 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:10.593 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:10.594 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:10.594 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.385+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:10.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.592+0000 7f886bb29640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:10.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.592+0000 7f886bb29640 -1 osd.10 97 *** Got signal Terminated *** 2024-09-05T01:35:10.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:10 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:10.592+0000 7f886bb29640 -1 osd.10 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:11.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:11.174+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:11.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:11.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:11 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:11.366+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:12.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:12 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:12.159+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:12.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:12.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:12 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:12.383+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:13.251 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:13 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:13.111+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:13.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:13.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:13.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:13.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:13.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:13.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:13.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:13 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:13.370+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:14.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:14 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:14.115+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:14.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:14.662 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:14 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:14.402+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:15.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:15 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:15.070+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:15.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:52.299705+0000 front 2024-09-05T01:33:52.299804+0000 (oldest deadline 2024-09-05T01:34:15.199673+0000) 2024-09-05T01:35:15.911 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:58.700459+0000 front 2024-09-05T01:33:58.700619+0000 (oldest deadline 2024-09-05T01:34:27.501465+0000) 2024-09-05T01:35:15.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:09.802920+0000 front 2024-09-05T01:34:09.802757+0000 (oldest deadline 2024-09-05T01:34:33.902616+0000) 2024-09-05T01:35:15.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6814 osd.3 since back 2024-09-05T01:34:16.803660+0000 front 2024-09-05T01:34:16.803557+0000 (oldest deadline 2024-09-05T01:34:41.503651+0000) 2024-09-05T01:35:15.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:25.905833+0000 front 2024-09-05T01:34:25.905886+0000 (oldest deadline 2024-09-05T01:34:51.206010+0000) 2024-09-05T01:35:15.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:32.907228+0000 front 2024-09-05T01:34:32.907256+0000 (oldest deadline 2024-09-05T01:34:56.407000+0000) 2024-09-05T01:35:15.912 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:15 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10[59270]: 2024-09-05T01:35:15.428+0000 7f8864137640 -1 osd.10 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:39.307916+0000 front 2024-09-05T01:34:39.307990+0000 (oldest deadline 2024-09-05T01:35:06.308273+0000) 2024-09-05T01:35:16.400 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:16 smithi183 podman[70804]: 2024-09-05 01:35:16.121256525 +0000 UTC m=+5.644699006 container died c26550dc7e008658ee353714a78bdbb02b42dde1412f8d387f5a5ac1abfd2600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:16.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:16 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:16.098+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:16.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:16 smithi183 podman[70804]: 2024-09-05 01:35:16.507312319 +0000 UTC m=+6.030754799 container cleanup c26550dc7e008658ee353714a78bdbb02b42dde1412f8d387f5a5ac1abfd2600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e) 2024-09-05T01:35:16.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:16 smithi183 bash[70804]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10 2024-09-05T01:35:17.117 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:16 smithi183 podman[70820]: 2024-09-05 01:35:16.76695171 +0000 UTC m=+0.639385672 container remove c26550dc7e008658ee353714a78bdbb02b42dde1412f8d387f5a5ac1abfd2600 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:35:17.370 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.019517362 +0000 UTC m=+0.015918589 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:35:17.370 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.12600696 +0000 UTC m=+0.122408173 container create 0616841757f404984b5d52651d63437065b820ed2e78f1878067c566308fbde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:17.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:17 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:17.106+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:17.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.369800845 +0000 UTC m=+0.366202073 container init 0616841757f404984b5d52651d63437065b820ed2e78f1878067c566308fbde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-05T01:35:17.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.373026685 +0000 UTC m=+0.369427902 container start 0616841757f404984b5d52651d63437065b820ed2e78f1878067c566308fbde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-09-05T01:35:17.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.453083289 +0000 UTC m=+0.449484512 container attach 0616841757f404984b5d52651d63437065b820ed2e78f1878067c566308fbde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T01:35:17.661 INFO:journalctl@ceph.osd.10.smithi183.stdout:Sep 05 01:35:17 smithi183 podman[70908]: 2024-09-05 01:35:17.55999261 +0000 UTC m=+0.556393825 container died 0616841757f404984b5d52651d63437065b820ed2e78f1878067c566308fbde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-05T01:35:18.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.10.service' 2024-09-05T01:35:18.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:18.239 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-05T01:35:18.239 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-05T01:35:18.239 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.11 2024-09-05T01:35:18.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.110+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:18.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 systemd[1]: Stopping Ceph osd.11 for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:19.000 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.574+0000 7f01a27aa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:19.001 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.574+0000 7f01a27aa640 -1 osd.11 97 *** Got signal Terminated *** 2024-09-05T01:35:19.001 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:18 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:18.574+0000 7f01a27aa640 -1 osd.11 97 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T01:35:19.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:19.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:19 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:19.123+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:20.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:20 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:20.118+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:21.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:21 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:21.100+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:22.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:22.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:22 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:22.094+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-09-05T01:34:58.992479+0000 front 2024-09-05T01:34:58.992500+0000 (oldest deadline 2024-09-05T01:35:21.892530+0000) 2024-09-05T01:35:23.500 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6806 osd.0 since back 2024-09-05T01:33:48.178803+0000 front 2024-09-05T01:33:48.179086+0000 (oldest deadline 2024-09-05T01:34:19.979434+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-09-05T01:33:59.979953+0000 front 2024-09-05T01:33:59.979829+0000 (oldest deadline 2024-09-05T01:34:25.880017+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6804 osd.2 since back 2024-09-05T01:34:10.581080+0000 front 2024-09-05T01:34:10.580991+0000 (oldest deadline 2024-09-05T01:34:31.681131+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-09-05T01:34:26.085331+0000 front 2024-09-05T01:34:26.085384+0000 (oldest deadline 2024-09-05T01:34:48.385413+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6812 osd.5 since back 2024-09-05T01:34:31.687706+0000 front 2024-09-05T01:34:31.687643+0000 (oldest deadline 2024-09-05T01:34:56.387809+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.129:6822 osd.6 since back 2024-09-05T01:34:37.989644+0000 front 2024-09-05T01:34:37.989656+0000 (oldest deadline 2024-09-05T01:35:09.190077+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-09-05T01:34:49.190852+0000 front 2024-09-05T01:34:49.190783+0000 (oldest deadline 2024-09-05T01:35:14.490782+0000) 2024-09-05T01:35:23.501 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:23 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11[60404]: 2024-09-05T01:35:23.136+0000 7f019adb8640 -1 osd.11 97 heartbeat_check: no reply from 172.21.15.194:6820 osd.8 since back 2024-09-05T01:34:58.992479+0000 front 2024-09-05T01:34:58.992500+0000 (oldest deadline 2024-09-05T01:35:21.892530+0000) 2024-09-05T01:35:24.317 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:24 smithi194 podman[72401]: 2024-09-05 01:35:24.055325259 +0000 UTC m=+5.619914805 container died 48dc2dc829c9acafc1acd341cdc7314da5fff15fc3efcf5f7f9548a5a2e5e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e) 2024-09-05T01:35:24.701 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:24 smithi194 podman[72401]: 2024-09-05 01:35:24.416854868 +0000 UTC m=+5.981444415 container cleanup 48dc2dc829c9acafc1acd341cdc7314da5fff15fc3efcf5f7f9548a5a2e5e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:35:24.701 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:24 smithi194 bash[72401]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11 2024-09-05T01:35:25.000 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:24 smithi194 podman[72416]: 2024-09-05 01:35:24.70124725 +0000 UTC m=+0.640030076 container remove 48dc2dc829c9acafc1acd341cdc7314da5fff15fc3efcf5f7f9548a5a2e5e783 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:35:25.271 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:24.913057768 +0000 UTC m=+0.015852868 image pull 77770cf7e895c297c29130881c012db5698948fe2279610297198415e1f3d0ac quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067 2024-09-05T01:35:25.271 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:25.026922509 +0000 UTC m=+0.129717606 container create 15a843c1465a40070f656ad2e5ffc5f5fb914652f65637cbeaeed7b9ecaa04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T01:35:25.271 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:25.270927006 +0000 UTC m=+0.373722100 container init 15a843c1465a40070f656ad2e5ffc5f5fb914652f65637cbeaeed7b9ecaa04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e) 2024-09-05T01:35:25.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:25.274446508 +0000 UTC m=+0.377241605 container start 15a843c1465a40070f656ad2e5ffc5f5fb914652f65637cbeaeed7b9ecaa04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:35:25.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:25.354155854 +0000 UTC m=+0.456950965 container attach 15a843c1465a40070f656ad2e5ffc5f5fb914652f65637cbeaeed7b9ecaa04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-05T01:35:25.751 INFO:journalctl@ceph.osd.11.smithi194.stdout:Sep 05 01:35:25 smithi194 podman[72505]: 2024-09-05 01:35:25.461064574 +0000 UTC m=+0.563859677 container died 15a843c1465a40070f656ad2e5ffc5f5fb914652f65637cbeaeed7b9ecaa04a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T01:35:26.054 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@osd.11.service' 2024-09-05T01:35:26.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:26.096 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-05T01:35:26.096 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-05T01:35:26.096 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.a 2024-09-05T01:35:26.389 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:26 smithi129 systemd[1]: Stopping Ceph mds.a for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:26.647 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:26 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a[71608]: 2024-09-05T01:35:26.388+0000 7f61d0eca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:26.647 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:26 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a[71608]: 2024-09-05T01:35:26.388+0000 7f61d0eca640 -1 mds.a *** got signal Terminated *** 2024-09-05T01:35:29.732 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:29 smithi129 podman[109329]: 2024-09-05 01:35:29.454611648 +0000 UTC m=+3.167111132 container died f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-05T01:35:29.992 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:29 smithi129 podman[109329]: 2024-09-05 01:35:29.829895808 +0000 UTC m=+3.542395289 container cleanup f96e531a7bcab25460b3db1131ee97bfff51e3f5181d4dd6e25f810ec0cb9a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:35:29.992 INFO:journalctl@ceph.mds.a.smithi129.stdout:Sep 05 01:35:29 smithi129 bash[109329]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-a 2024-09-05T01:35:30.255 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.a.service' 2024-09-05T01:35:30.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:30.290 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-05T01:35:30.290 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-05T01:35:30.291 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.d 2024-09-05T01:35:30.492 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:30 smithi129 systemd[1]: Stopping Ceph mds.d for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:30.992 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d[70402]: 2024-09-05T01:35:30.608+0000 7f5f6d515640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:30.992 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:30 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d[70402]: 2024-09-05T01:35:30.608+0000 7f5f6d515640 -1 mds.d *** got signal Terminated *** 2024-09-05T01:35:34.918 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:34 smithi129 podman[109463]: 2024-09-05 01:35:34.492586435 +0000 UTC m=+3.983584704 container died 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e) 2024-09-05T01:35:35.226 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:35 smithi129 podman[109463]: 2024-09-05 01:35:35.000464335 +0000 UTC m=+4.491462607 container cleanup 4ad15dc01ebed47084bdb0c530792b6d242457d2ff2adc1dc2ebba225b391be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-05T01:35:35.227 INFO:journalctl@ceph.mds.d.smithi129.stdout:Sep 05 01:35:35 smithi129 bash[109463]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-d 2024-09-05T01:35:35.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.d.service' 2024-09-05T01:35:35.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:35.404 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-05T01:35:35.404 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-05T01:35:35.404 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.g 2024-09-05T01:35:35.742 INFO:journalctl@ceph.mds.g.smithi129.stdout:Sep 05 01:35:35 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-g[71284]: 2024-09-05T01:35:35.728+0000 7fba85288640 -1 mds.g *** got signal Terminated *** 2024-09-05T01:35:39.071 INFO:journalctl@ceph.mds.g.smithi129.stdout:Sep 05 01:35:38 smithi129 podman[109584]: 2024-09-05 01:35:38.761869757 +0000 UTC m=+3.135295126 container died 47e93a50d05328e16b4f754172a268e6d62ce835ab5418f87133181dcff8402b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-g, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T01:35:39.405 INFO:journalctl@ceph.mds.g.smithi129.stdout:Sep 05 01:35:39 smithi129 podman[109584]: 2024-09-05 01:35:39.154105806 +0000 UTC m=+3.527531174 container cleanup 47e93a50d05328e16b4f754172a268e6d62ce835ab5418f87133181dcff8402b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-g, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T01:35:39.405 INFO:journalctl@ceph.mds.g.smithi129.stdout:Sep 05 01:35:39 smithi129 bash[109584]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-g 2024-09-05T01:35:39.530 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.g.service' 2024-09-05T01:35:39.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:39.563 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-05T01:35:39.563 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-05T01:35:39.563 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.j 2024-09-05T01:35:39.742 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:39 smithi129 systemd[1]: Stopping Ceph mds.j for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:40.492 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j[69999]: 2024-09-05T01:35:39.999+0000 7f64d30f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:40.492 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:40 smithi129 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j[69999]: 2024-09-05T01:35:39.999+0000 7f64d30f6640 -1 mds.j *** got signal Terminated *** 2024-09-05T01:35:42.926 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:42 smithi129 podman[109702]: 2024-09-05 01:35:42.642566773 +0000 UTC m=+2.866547183 container died efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-91c8a7e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-05T01:35:43.242 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:42 smithi129 podman[109702]: 2024-09-05 01:35:42.999590481 +0000 UTC m=+3.223570878 container cleanup efa435e5ea7f484e59b71853ef63941e92788210ab57bf2fa35c5fdaa16d1900 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-05T01:35:43.242 INFO:journalctl@ceph.mds.j.smithi129.stdout:Sep 05 01:35:42 smithi129 bash[109702]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-j 2024-09-05T01:35:43.381 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.j.service' 2024-09-05T01:35:43.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:43.413 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-05T01:35:43.413 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-05T01:35:43.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.b 2024-09-05T01:35:43.715 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:43 smithi183 systemd[1]: Stopping Ceph mds.b for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:43.973 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: 2024-09-05T01:35:43.714+0000 7f5467542640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:43.973 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:43 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b[64612]: 2024-09-05T01:35:43.714+0000 7f5467542640 -1 mds.b *** got signal Terminated *** 2024-09-05T01:35:46.049 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:45 smithi183 podman[71012]: 2024-09-05 01:35:45.742127281 +0000 UTC m=+2.137137558 container died 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T01:35:46.374 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:46 smithi183 podman[71012]: 2024-09-05 01:35:46.14773756 +0000 UTC m=+2.542747878 container cleanup 78616fca170d3aab53d6f08ac5767aa468dd058b3fe72d3890625aebf85c5828 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-91c8a7e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-05T01:35:46.374 INFO:journalctl@ceph.mds.b.smithi183.stdout:Sep 05 01:35:46 smithi183 bash[71012]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-b 2024-09-05T01:35:46.522 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.b.service' 2024-09-05T01:35:46.558 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:46.559 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-05T01:35:46.559 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-05T01:35:46.559 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.e 2024-09-05T01:35:46.911 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:46 smithi183 systemd[1]: Stopping Ceph mds.e for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:46.911 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: 2024-09-05T01:35:46.876+0000 7f620ead6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:46.911 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:46 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e[64008]: 2024-09-05T01:35:46.876+0000 7f620ead6640 -1 mds.e *** got signal Terminated *** 2024-09-05T01:35:50.068 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:49 smithi183 podman[71146]: 2024-09-05 01:35:49.756190591 +0000 UTC m=+2.981042278 container died 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-05T01:35:50.068 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:50 smithi183 podman[71146]: 2024-09-05 01:35:50.067586351 +0000 UTC m=+3.292438050 container cleanup 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-91c8a7e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T01:35:50.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:50 smithi183 bash[71146]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e 2024-09-05T01:35:50.411 INFO:journalctl@ceph.mds.e.smithi183.stdout:Sep 05 01:35:50 smithi183 podman[71157]: 2024-09-05 01:35:50.26055132 +0000 UTC m=+0.504607194 container remove 93726fe80c75c8c8751ba059ac1acc5c8e515b28709d0177bed732e6f121c63d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T01:35:50.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.e.service' 2024-09-05T01:35:50.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:50.542 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-05T01:35:50.542 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-05T01:35:50.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.h 2024-09-05T01:35:50.855 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:50 smithi183 systemd[1]: Stopping Ceph mds.h for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:51.161 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: 2024-09-05T01:35:50.854+0000 7f7477bb6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:51.161 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:50 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h[64311]: 2024-09-05T01:35:50.854+0000 7f7477bb6640 -1 mds.h *** got signal Terminated *** 2024-09-05T01:35:54.161 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:53 smithi183 podman[71263]: 2024-09-05 01:35:53.754296085 +0000 UTC m=+2.995966211 container died 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T01:35:54.589 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:54 smithi183 podman[71263]: 2024-09-05 01:35:54.396181311 +0000 UTC m=+3.637851437 container cleanup 00039077e679451e7711c17ffed82b3e21916b0c7c3a271748732a0dcbca649a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-05T01:35:54.589 INFO:journalctl@ceph.mds.h.smithi183.stdout:Sep 05 01:35:54 smithi183 bash[71263]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-h 2024-09-05T01:35:54.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.h.service' 2024-09-05T01:35:54.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:54.833 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-05T01:35:54.833 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-05T01:35:54.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.k 2024-09-05T01:35:55.158 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:54 smithi183 systemd[1]: Stopping Ceph mds.k for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:35:55.411 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k[63698]: 2024-09-05T01:35:55.157+0000 7fcb65309640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:35:55.411 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:55 smithi183 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k[63698]: 2024-09-05T01:35:55.157+0000 7fcb65309640 -1 mds.k *** got signal Terminated *** 2024-09-05T01:35:59.443 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:59 smithi183 podman[71385]: 2024-09-05 01:35:59.162568001 +0000 UTC m=+4.108896534 container died 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-05T01:35:59.792 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:59 smithi183 podman[71385]: 2024-09-05 01:35:59.540865253 +0000 UTC m=+4.487193783 container cleanup 498078ee559a8595fc1fa296778580f280457efccc15d504ad095f03340be678 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T01:35:59.792 INFO:journalctl@ceph.mds.k.smithi183.stdout:Sep 05 01:35:59 smithi183 bash[71385]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-k 2024-09-05T01:35:59.915 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.k.service' 2024-09-05T01:35:59.946 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:35:59.946 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-05T01:35:59.947 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-05T01:35:59.947 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.c 2024-09-05T01:36:00.249 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:35:59 smithi194 systemd[1]: Stopping Ceph mds.c for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:36:00.500 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: 2024-09-05T01:36:00.247+0000 7f0df5fae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:36:00.500 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:00 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c[65373]: 2024-09-05T01:36:00.247+0000 7f0df5fae640 -1 mds.c *** got signal Terminated *** 2024-09-05T01:36:02.198 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:01 smithi194 podman[72608]: 2024-09-05 01:36:01.752848868 +0000 UTC m=+1.609760162 container died ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:36:02.482 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 podman[72608]: 2024-09-05 01:36:02.297376386 +0000 UTC m=+2.154287665 container cleanup ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731) 2024-09-05T01:36:02.482 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 bash[72608]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c 2024-09-05T01:36:02.719 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.c.service' 2024-09-05T01:36:02.742 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 podman[72633]: 2024-09-05 01:36:02.481687513 +0000 UTC m=+0.725883142 container remove ee8363500991f55cf40c30b73d14b3705f7037e23a02f78e30658bdaa740abcb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-05T01:36:02.742 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-05T01:36:02.742 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 systemd[1]: Stopped Ceph mds.c for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:36:02.742 INFO:journalctl@ceph.mds.c.smithi194.stdout:Sep 05 01:36:02 smithi194 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.c.service: Consumed 5.358s CPU time. 2024-09-05T01:36:02.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:36:02.756 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-05T01:36:02.756 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-05T01:36:02.756 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.f 2024-09-05T01:36:03.001 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:02 smithi194 systemd[1]: Stopping Ceph mds.f for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:36:03.500 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: 2024-09-05T01:36:03.100+0000 7f759a750640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:36:03.501 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:03 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f[65076]: 2024-09-05T01:36:03.100+0000 7f759a750640 -1 mds.f *** got signal Terminated *** 2024-09-05T01:36:06.019 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:05 smithi194 podman[72744]: 2024-09-05 01:36:05.753618045 +0000 UTC m=+2.781663210 container died f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-91c8a7e) 2024-09-05T01:36:06.393 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:06 smithi194 podman[72744]: 2024-09-05 01:36:06.117604301 +0000 UTC m=+3.145649473 container cleanup f6598336e447a8e654dc781bdb9979b2128c6ac906951e78bdeff3c05e734b8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-91c8a7e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-05T01:36:06.394 INFO:journalctl@ceph.mds.f.smithi194.stdout:Sep 05 01:36:06 smithi194 bash[72744]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-f 2024-09-05T01:36:06.544 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.f.service' 2024-09-05T01:36:06.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:36:06.579 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-05T01:36:06.579 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-05T01:36:06.579 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.i 2024-09-05T01:36:06.947 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:06 smithi194 systemd[1]: Stopping Ceph mds.i for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:36:07.250 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i[65672]: 2024-09-05T01:36:06.946+0000 7f532c7d6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:36:07.250 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:06 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i[65672]: 2024-09-05T01:36:06.946+0000 7f532c7d6640 -1 mds.i *** got signal Terminated *** 2024-09-05T01:36:10.088 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:09 smithi194 podman[72862]: 2024-09-05 01:36:09.750596472 +0000 UTC m=+2.951875025 container died ab1fbeabe1465c2917d69d126ee05c1bb786e82482fba9e5e56729026519c9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:36:10.439 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:10 smithi194 podman[72862]: 2024-09-05 01:36:10.170742405 +0000 UTC m=+3.372020954 container cleanup ab1fbeabe1465c2917d69d126ee05c1bb786e82482fba9e5e56729026519c9bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-91c8a7e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T01:36:10.439 INFO:journalctl@ceph.mds.i.smithi194.stdout:Sep 05 01:36:10 smithi194 bash[72862]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-i 2024-09-05T01:36:10.653 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.i.service' 2024-09-05T01:36:10.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:36:10.686 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-05T01:36:10.686 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-05T01:36:10.686 DEBUG:teuthology.orchestra.run.smithi194:> sudo systemctl stop ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.l 2024-09-05T01:36:11.000 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:10 smithi194 systemd[1]: Stopping Ceph mds.l for c8062bdc-6b22-11ef-bcd6-c7b262605968... 2024-09-05T01:36:11.500 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l[64773]: 2024-09-05T01:36:11.006+0000 7f2703791640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T01:36:11.500 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:11 smithi194 ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l[64773]: 2024-09-05T01:36:11.006+0000 7f2703791640 -1 mds.l *** got signal Terminated *** 2024-09-05T01:36:13.367 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 podman[72983]: 2024-09-05 01:36:13.008951063 +0000 UTC m=+2.110225227 container died 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-91c8a7e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:36:13.642 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 podman[72983]: 2024-09-05 01:36:13.457673553 +0000 UTC m=+2.558947715 container cleanup 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-91c8a7e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T01:36:13.643 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 bash[72983]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l 2024-09-05T01:36:13.877 DEBUG:teuthology.orchestra.run.smithi194:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.l.service' 2024-09-05T01:36:13.898 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 podman[72996]: 2024-09-05 01:36:13.641900141 +0000 UTC m=+0.527591074 container remove 5c07347ae7796c7a4ebe5ac78b4aa3efac9a55fc59edc851a193e4a495796d22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:bc24b8b0fb57fbf3f589c6abdf5aefd5f62db62238cebcb4209974c168290067, name=ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-91c8a7e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T01:36:13.898 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.l.service: Deactivated successfully. 2024-09-05T01:36:13.898 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 systemd[1]: Stopped Ceph mds.l for c8062bdc-6b22-11ef-bcd6-c7b262605968. 2024-09-05T01:36:13.898 INFO:journalctl@ceph.mds.l.smithi194.stdout:Sep 05 01:36:13 smithi194 systemd[1]: ceph-c8062bdc-6b22-11ef-bcd6-c7b262605968@mds.l.service: Consumed 1.115s CPU time. 2024-09-05T01:36:13.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T01:36:13.909 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-05T01:36:13.910 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T01:36:14.118 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:36:16.703 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T01:36:16.905 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:36:19.431 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T01:36:19.633 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:36:22.064 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:36:22.097 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:36:22.131 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T01:36:22.158 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-05T01:36:22.160 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi129/crash 2024-09-05T01:36:22.161 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T01:36:22.201 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T01:36:22.201 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-09-05T01:36:22.203 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi183/crash 2024-09-05T01:36:22.204 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T01:36:22.245 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T01:36:22.245 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-09-05T01:36:22.247 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi194/crash 2024-09-05T01:36:22.248 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T01:36:22.285 INFO:teuthology.orchestra.run.smithi194.stderr:tar: /var/lib/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T01:36:22.285 INFO:teuthology.orchestra.run.smithi194.stderr:tar: Error is not recoverable: exiting now 2024-09-05T01:36:22.286 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-05T01:36:22.287 DEBUG:teuthology.orchestra.run.smithi129:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-05T01:36:22.395 INFO:tasks.cephadm:Compressing logs... 2024-09-05T01:36:22.395 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T01:36:22.398 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T01:36:22.401 DEBUG:teuthology.orchestra.run.smithi194:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T01:36:22.434 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T01:36:22.436 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.10.log 2024-09-05T01:36:22.436 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.1.log 2024-09-05T01:36:22.443 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.7.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.10.log: /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.b.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.k.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.4.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.y.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.h.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.b.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.e.log 2024-09-05T01:36:22.444 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T01:36:22.446 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr: /var/log/ceph/ceph-client.admin.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.0.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.9.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.a.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.a.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.g.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.6.log 2024-09-05T01:36:22.447 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.d.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.x.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.j.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.3.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:gzip/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.3.log: -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.a.log.gz 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr: 86.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr: 94.5% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.g.log.gz 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr: 94.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.d.log.gz 2024-09-05T01:36:22.448 INFO:teuthology.orchestra.run.smithi129.stderr: 94.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.j.log.gz 2024-09-05T01:36:22.451 INFO:teuthology.orchestra.run.smithi194.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T01:36:22.451 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T01:36:22.451 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T01:36:22.451 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.i.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log: /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.c.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose --/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.i.log: /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.11.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.2.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.5.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.l.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.5.log: /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.8.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.z.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:gzip/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.8.log: -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.c.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.f.log 2024-09-05T01:36:22.452 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T01:36:22.453 INFO:teuthology.orchestra.run.smithi129.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-05T01:36:22.481 INFO:teuthology.orchestra.run.smithi129.stderr: 83.1% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T01:36:22.484 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T01:36:22.493 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T01:36:22.530 INFO:teuthology.orchestra.run.smithi183.stderr: 86.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T01:36:22.553 INFO:teuthology.orchestra.run.smithi194.stderr:/var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.z.log.gz 2024-09-05T01:36:22.563 INFO:teuthology.orchestra.run.smithi183.stderr: 94.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.k.log.gz 2024-09-05T01:36:22.567 INFO:teuthology.orchestra.run.smithi183.stderr: 90.5% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T01:36:22.589 INFO:teuthology.orchestra.run.smithi183.stderr: 93.2% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.y.log.gz 2024-09-05T01:36:22.610 INFO:teuthology.orchestra.run.smithi183.stderr: 83.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T01:36:22.616 INFO:teuthology.orchestra.run.smithi183.stderr: 91.0% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T01:36:22.635 INFO:teuthology.orchestra.run.smithi194.stderr: 90.5% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T01:36:22.644 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T01:36:22.645 INFO:teuthology.orchestra.run.smithi183.stderr: 94.5% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.e.log.gz 2024-09-05T01:36:22.778 INFO:teuthology.orchestra.run.smithi129.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T01:36:22.794 INFO:teuthology.orchestra.run.smithi194.stderr: 83.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T01:36:23.011 INFO:teuthology.orchestra.run.smithi194.stderr: 86.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T01:36:23.020 INFO:teuthology.orchestra.run.smithi194.stderr: 94.6% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.l.log.gz 2024-09-05T01:36:23.153 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.h.log.gz 2024-09-05T01:36:23.172 INFO:teuthology.orchestra.run.smithi194.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T01:36:23.176 INFO:teuthology.orchestra.run.smithi194.stderr: 91.0% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T01:36:23.663 INFO:teuthology.orchestra.run.smithi194.stderr: 93.7% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.i.log.gz 2024-09-05T01:36:24.548 INFO:teuthology.orchestra.run.smithi183.stderr: 91.7% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.b.log.gz 2024-09-05T01:36:24.932 INFO:teuthology.orchestra.run.smithi194.stderr: 94.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.f.log.gz 2024-09-05T01:36:25.507 INFO:teuthology.orchestra.run.smithi194.stderr: 95.2% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.c.log.gz 2024-09-05T01:36:26.410 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mgr.x.log.gz 2024-09-05T01:36:26.604 INFO:teuthology.orchestra.run.smithi194.stderr: 91.7% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.c.log.gz 2024-09-05T01:36:37.022 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mon.a.log.gz 2024-09-05T01:37:09.716 INFO:teuthology.orchestra.run.smithi183.stderr: 95.3% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-mds.b.log.gz 2024-09-05T01:37:35.258 INFO:teuthology.orchestra.run.smithi194.stderr: 90.9% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.11.log.gz 2024-09-05T01:38:00.861 INFO:teuthology.orchestra.run.smithi183.stderr: 90.9% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.10.log.gz 2024-09-05T01:38:30.480 INFO:teuthology.orchestra.run.smithi183.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.1.log.gz 2024-09-05T01:38:53.483 INFO:teuthology.orchestra.run.smithi129.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-05T01:38:54.060 INFO:teuthology.orchestra.run.smithi183.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.7.log.gz 2024-09-05T01:39:22.284 INFO:teuthology.orchestra.run.smithi183.stderr: 90.9% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.4.log.gz 2024-09-05T01:39:22.286 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-09-05T01:39:22.286 INFO:teuthology.orchestra.run.smithi183.stderr:real 2m59.871s 2024-09-05T01:39:22.286 INFO:teuthology.orchestra.run.smithi183.stderr:user 4m56.077s 2024-09-05T01:39:22.286 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m10.381s 2024-09-05T01:39:35.544 INFO:teuthology.orchestra.run.smithi194.stderr: 91.0% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.2.log.gz 2024-09-05T01:39:42.194 INFO:teuthology.orchestra.run.smithi194.stderr: 90.9% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.8.log.gz 2024-09-05T01:40:18.108 INFO:teuthology.orchestra.run.smithi194.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.5.log.gz 2024-09-05T01:40:18.110 INFO:teuthology.orchestra.run.smithi194.stderr: 2024-09-05T01:40:18.110 INFO:teuthology.orchestra.run.smithi194.stderr:real 3m55.695s 2024-09-05T01:40:18.110 INFO:teuthology.orchestra.run.smithi194.stderr:user 5m9.125s 2024-09-05T01:40:18.110 INFO:teuthology.orchestra.run.smithi194.stderr:sys 0m11.103s 2024-09-05T01:45:09.704 INFO:teuthology.orchestra.run.smithi129.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.3.log.gz 2024-09-05T01:47:16.836 INFO:teuthology.orchestra.run.smithi129.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.0.log.gz 2024-09-05T01:49:57.519 INFO:teuthology.orchestra.run.smithi129.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.6.log.gz 2024-09-05T01:50:24.152 INFO:teuthology.orchestra.run.smithi129.stderr: 90.8% -- replaced with /var/log/ceph/c8062bdc-6b22-11ef-bcd6-c7b262605968/ceph-osd.9.log.gz 2024-09-05T01:50:24.155 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-05T01:50:24.155 INFO:teuthology.orchestra.run.smithi129.stderr:real 14m1.743s 2024-09-05T01:50:24.155 INFO:teuthology.orchestra.run.smithi129.stderr:user 7m11.857s 2024-09-05T01:50:24.155 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m18.707s 2024-09-05T01:50:24.155 INFO:tasks.cephadm:Archiving logs... 2024-09-05T01:50:24.156 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi129/log 2024-09-05T01:50:24.157 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T01:51:37.366 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi183/log 2024-09-05T01:51:37.367 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T01:52:27.999 DEBUG:teuthology.misc:Transferring archived files from smithi194:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi194/log 2024-09-05T01:52:28.000 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T01:53:23.603 INFO:tasks.cephadm:Removing cluster... 2024-09-05T01:53:23.603 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force 2024-09-05T01:53:23.827 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:53:24.577 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force 2024-09-05T01:53:24.782 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:53:25.444 DEBUG:teuthology.orchestra.run.smithi194:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid c8062bdc-6b22-11ef-bcd6-c7b262605968 --force 2024-09-05T01:53:25.648 INFO:teuthology.orchestra.run.smithi194.stdout:Deleting cluster with fsid: c8062bdc-6b22-11ef-bcd6-c7b262605968 2024-09-05T01:53:26.314 INFO:tasks.cephadm:Removing cephadm ... 2024-09-05T01:53:26.314 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T01:53:26.333 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T01:53:26.351 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T01:53:26.367 INFO:tasks.cephadm:Teardown complete 2024-09-05T01:53:26.367 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T01:53:26.378 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T01:53:26.379 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T01:53:26.381 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T01:53:26.394 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T01:53:26.519 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T01:53:26.529 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T01:53:26.529 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:53:26.532 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:53:26.535 DEBUG:teuthology.orchestra.run.smithi194:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T01:53:26.548 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:53:26.549 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:53:26.549 INFO:teuthology.orchestra.run.smithi194.stderr:bash: line 1: ntpq: command not found 2024-09-05T01:53:26.554 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:53:26.554 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-05T01:53:26.554 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 +183us[ +180us] +/- 61ms 2024-09-05T01:53:26.554 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 65 -466us[ -469us] +/- 59ms 2024-09-05T01:53:26.554 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 +348us[ +345us] +/- 69ms 2024-09-05T01:53:26.555 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:53:26.564 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:53:26.565 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-09-05T01:53:26.565 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +189us[ +200us] +/- 61ms 2024-09-05T01:53:26.565 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 -471us[ -459us] +/- 59ms 2024-09-05T01:53:26.565 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +343us[ +343us] +/- 69ms 2024-09-05T01:53:26.566 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:53:26.567 INFO:teuthology.orchestra.run.smithi194.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T01:53:26.567 INFO:teuthology.orchestra.run.smithi194.stdout:=============================================================================== 2024-09-05T01:53:26.567 INFO:teuthology.orchestra.run.smithi194.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 +194us[ +206us] +/- 62ms 2024-09-05T01:53:26.568 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -465us[ -453us] +/- 60ms 2024-09-05T01:53:26.568 INFO:teuthology.orchestra.run.smithi194.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +336us[ +349us] +/- 70ms 2024-09-05T01:53:26.568 INFO:teuthology.orchestra.run.smithi194.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T01:53:26.568 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T01:53:26.586 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T01:53:26.587 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T01:53:26.604 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T01:53:26.778 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T01:53:26.897 DEBUG:teuthology.orchestra.run.smithi194:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T01:53:26.987 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:53:27.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:27.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:53:27.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:27.051 DEBUG:teuthology.orchestra.run.smithi194:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T01:53:27.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:27.079 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T01:53:27.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T01:53:27.100 INFO:teuthology.task.internal:Duration was 3265.296985 seconds 2024-09-05T01:53:27.101 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T01:53:27.111 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T01:53:27.111 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T01:53:27.114 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T01:53:27.116 DEBUG:teuthology.orchestra.run.smithi194:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T01:53:27.166 INFO:teuthology.orchestra.run.smithi194.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:53:27.185 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:53:27.211 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T01:53:27.551 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T01:53:27.551 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-05T01:53:27.552 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T01:53:27.576 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-09-05T01:53:27.576 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T01:53:27.603 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi194.front.sepia.ceph.com 2024-09-05T01:53:27.603 DEBUG:teuthology.orchestra.run.smithi194:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T01:53:27.666 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T01:53:27.666 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T01:53:27.669 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T01:53:27.672 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T01:53:27.699 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T01:53:27.699 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T01:53:27.740 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T01:53:27.743 DEBUG:teuthology.orchestra.run.smithi194:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T01:53:28.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T01:53:28.581 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T01:53:28.581 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T01:53:28.623 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T01:53:28.667 DEBUG:teuthology.orchestra.run.smithi194:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T01:53:28.704 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T01:53:28.715 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.717 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.720 DEBUG:teuthology.orchestra.run.smithi194:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.745 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-09-05T01:53:28.752 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-05T01:53:28.774 INFO:teuthology.orchestra.run.smithi194.stdout:kernel.core_pattern = core 2024-09-05T01:53:28.817 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:28.833 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:28.848 DEBUG:teuthology.orchestra.run.smithi194:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T01:53:28.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T01:53:28.873 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T01:53:28.885 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T01:53:28.886 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi129 2024-09-05T01:53:28.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T01:53:28.941 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi183 2024-09-05T01:53:28.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T01:53:28.980 DEBUG:teuthology.misc:Transferring archived files from smithi194:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-04_21:24:16-fs-squid-distro-default-smithi/7889754/remote/smithi194 2024-09-05T01:53:28.980 DEBUG:teuthology.orchestra.run.smithi194:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T01:53:29.019 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T01:53:29.019 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T01:53:29.022 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T01:53:29.024 DEBUG:teuthology.orchestra.run.smithi194:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T01:53:29.068 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T01:53:29.082 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T01:53:29.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T01:53:29.093 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T01:53:29.094 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T01:53:29.096 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T01:53:29.098 DEBUG:teuthology.orchestra.run.smithi194:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T01:53:29.108 INFO:teuthology.orchestra.run.smithi129.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 01:53 /home/ubuntu/cephtest 2024-09-05T01:53:29.111 INFO:teuthology.orchestra.run.smithi183.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 01:53 /home/ubuntu/cephtest 2024-09-05T01:53:29.124 INFO:teuthology.orchestra.run.smithi194.stdout: 262217 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 01:53 /home/ubuntu/cephtest 2024-09-05T01:53:29.132 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T01:53:29.144 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T01:53:29.172 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 3265.2969851493835 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-05T01:53:29.172 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T01:53:29.253 INFO:teuthology.run:pass